非完全信息下无人机集群对抗研究综述

薛健; 赵琳; 向贤财; 吕科; 宏晨; 张宝琳; 岩延; 王泳

doi:10.11999/JEIT230544

非完全信息下无人机集群对抗研究综述

doi: 10.11999/JEIT230544

薛健¹,
赵琳¹,
向贤财¹,
吕科^{1, 2},
宏晨³,
张宝琳⁴,
岩延⁵,
王泳^5, ,

1.
中国科学院大学工程科学学院北京 100049
2.
鹏程实验室深圳 518055
3.
北京联合大学机器人学院北京 100101
4.
青岛科技大学自动化与电子工程学院青岛 266061
5.
中国科学院大学人工智能学院北京 100190

基金项目: 国家重点研发计划(2018AAA0100804)

详细信息

作者简介:
薛健：男，教授，研究方向为多智能体系统、模式识别和机器视觉

赵琳：女，博士生，研究方向为深度强化学习、无人机集群系统、博弈论

向贤财：男，硕士生，研究方向为深度强化学习、无人机集群系统

吕科：男，教授，研究方向为人工智能、计算机视觉

宏晨：男，副教授，研究方向为深度强化学习、无人机集群控制

张宝琳：男，教授，研究方向为无人机集群控制

岩延：男，副教授，研究方向为深度强化学习

王泳：男，讲师，研究方向为复杂系统建模与优化、模式识别、数据挖掘

通讯作者:
王泳　wangyong@ucas.ac.cn

中图分类号: TN975; TP391; TP181; V279; V249
计量
- 文章访问数: 1599
- HTML全文浏览量: 769
- PDF下载量: 460
- 被引次数: 0
出版历程
- 收稿日期: 2023-06-02
- 修回日期: 2023-09-28
- 网络出版日期: 2023-10-16
- 刊出日期: 2024-04-24

A Review of the Research on UAV Swarm Confrontation under Incomplete Information

XUE Jian¹,
ZHAO Lin¹,
XIANG Xiancai¹,
LÜ Ke^{1, 2},
HONG Chen³,
ZHANG Baolin⁴,
YAN Yan⁵,
WANG Yong^{5
, ,}

1.
School of Engineering Science, University of Chinese Academy of Sciences, Beijing 100049, China
2.
Peng Cheng Laboratory, Shenzhen 518055, China
3.
College of Robotics, Beijing Union University, Beijing 100101, China
4.
College of Automation and Electronic Engineering, Qingdao University of Science and Technology, Qingdao 266061, China
5.
College of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100190, China

Funds: The National Key Research and Development Program of China (2018AAA0100804)

摘要

摘要: 无人机集群以其具备的应用优势及发展前景，成为当前人工智能领域研究者关注的热点之一。而非完全信息下的无人机集群对抗技术，因其集群结构变化的高动态性以及环境信息复杂多变且不能完全感知的特点，成为对集群协同性与智能性要求最高的研究方向之一。其研究成果可以促进智能化无人系统的快速发展和广泛应用。该文全面回顾了非完全信息环境下无人机集群对抗研究的最新进展，按照包以德循环理论的思路将无人机集群对抗过程划分为态势评估、意图推断、任务规划与机动决策4个相互衔接的关键组成部分，并进一步将其细分为8个子研究目标。通过分析比较近年来的相关研究，着重阐述了无人机集群对抗领域各项任务的研究重点和难点以及已取得的成果，并讨论了无人机集群对抗技术所面临的挑战，包括大规模异构集群的协同控制、非完全信息的处理、复杂决策过程的建模以及实际应用任务的应对等。
- 无人机集群 /
- 非完全信息 /
- 态势评估 /
- 意图推断 /
- 任务规划 /
- 机动决策
Abstract: UAV (Unmanned Aerial Vehicle) swarm, with its application advantages and development prospects, has become one of the current hot spots of interest for researchers in the field of artificial intelligence. The UAV swarm confrontation technology under incomplete information has become one of the research directions with the highest requirements for swarm cooperativeness and intelligence due to the high dynamics of swarm structure changes and the complex and variable environmental information that cannot be fully perceived. Its research achievements can promote the rapid development and wide application of intelligent unmanned systems. This paper comprehensively reviews of the recent progress in the research of UAV swarm confrontation under incomplete information environments. According to the Observe-Orient-Decide-Act (OODA) loop theory, the UAV swarm confrontation process is divided into four interlocking key components of situation assessment, intention inference, mission planning, and maneuver decision, and is further subdivided into eight sub-research objectives. By analyzing and comparing the relevant research works in recent years, the research focuses and difficulties of various tasks in the field of UAV swarm confrontation and the achieved research results are highlighted, and the challenges faced by UAV swarm confrontation technology are discussed, including the cooperative control of large-scale heterogeneous swarms, the handling of incomplete information, the modeling of complex decision-making processes, and the tackling of practical application tasks.
- Unmanned Aerial Vehicle(UAV)swarm /
- Incomplete information /
- Situation assessment /
- Intention inference /
- Task planning /
- Maneuver decision

HTML全文

图 1 无人机集群对抗研究脉络

下载: 全尺寸图片幻灯片

图 2 无人机集群对抗态势评估研究框架

下载: 全尺寸图片幻灯片

图 3 无人机集群对抗中的意图推断研究框架

下载: 全尺寸图片幻灯片

图 4 无人机集群对抗机动决策研究框架

下载: 全尺寸图片幻灯片

表 1 非完全信息下无人机集群对抗具体研究内容

OODA循环	对抗循环环节	具体内容	具体含义
观察	态势评估	对抗态势评估	将态势定性分成优势、劣势、均势等，通过构造优势函数，利用双方状态信息进行态势评估，辅助集群决策
观察	态势评估	威胁因素评估	根据无人机特点，评估排序敌方目标和防空威胁，以辅助目标分配和决策
定位	意图推断	行为预测	利用已有信息推测敌机未来行为模式，预判使无人机集群获得决策优势
定位	意图推断	意图识别	对敌方集群当前行为意图进行定性判断，理解敌机意图使无人机集群获得决策优势
决策	任务规划	目标分配	考虑无人机的特性和任务需求，根据资源与优先级等约束，合理地分配目标，并实现集群内动态协同
决策	任务规划	航迹规划	根据任务需求，为无人机规划最佳路径，以安全到达目标
执行	机动决策	协同对抗	充分发挥集群力量，通过分散瓦解敌方防御，增大对敌方造成有效打击的可能性
执行	机动决策	追踪合围	对敌方的动态追踪与合围，通过限制敌方行动来保持我方优势与对敌方的压制

下载: 导出CSV

表 2 无人机集群空战对抗态势评估方法研究对比

参考文献	方法	仿真维度	无人机数量	影响因子权重	未来状态影响	非完全信息	异构集群
Wu等人^[6]	模糊推理	3D	1 vs. 1	客观计算	√	×	–
Zhang等人^[7]	模糊神经网络	3D	${n} $ vs. ${n} $	客观计算	√	×	×
Huang等人^[8]	贝叶斯推理	3D	1 vs. 1	客观计算	√	√	–
Xie等人^[9]	动态关联权值	3D	1 vs. 1	客观计算	√	×	–
高杨等人^[10]	改进态势模型	–	${n} $ vs. ${n} $	–	×	×	√
Shin等人^[12]	几何评分	3D	2 vs. 2	客观计算	×	×	×
√：考虑；×：不考虑；–：不涉及

下载: 导出CSV

表 3 无人机集群空战对抗中对敌方行为预测方法研究对比

参考文献	方法	仿真维度	非完全信息	目标的复杂运动	计算难度的降低	目标数量	研究目标
Pan等人^[26]	状态预测影响图	2D	√	×	×	多个	机动决策
Xi 等人^[27]	PSR-RBF	3D	×	×	×	单个	空战情况感知和威胁评估
Liu等人^[28]	GA-MASAC-TP Net	2D	×	√	×	多个	预测目标情况
Tan等人^[29]	Bi-LSTM	2D	×	×	×	单个	机动决策
Yang等人^[30]	MPC	3D	×	×	√	单个	机动决策
√: 考虑；×: 不考虑；–:不涉及

下载: 导出CSV

表 4 无人机集群目标分配方法研究对比

参考文献	方法	仿真维度	异构集群	非完全信息	目标威胁	资源约束	时间约束
严飞等人^[40]	CPN	2D	√	×	×	√	√
Zhen等人^[41]	改进CPN	2D	√	×	√	√	√
王峰等人^[42]	KnCMPSO	2D	√	×	×	√	√
Zhao等人^[43]	FTA	–	√	√	×	√	×
Jia等人^[44]	改进GA-随机规划	2D	√	√	×	√	√
赵玉亮等人^[45]	多策略融合粒子群	–	×	√	√	√	×
张安等人^[46]	IMSGWO	2D	√	√	√	×	√
Liu等人^[47]	MARL	–	×	√	×	√	√
√：考虑；×：不考虑；–：不涉及

下载: 导出CSV

表 5 无人机集群航迹规划方法研究对比

参考文献	方法	空间维度	异构集群	非完全信息	目标数量	环境威胁	防碰撞约束	时间协同约束
蔡星娟等人^[57]	NSGAIII-ICO	3D	×	×	单目标	√	√	√
左燕等人^[58]	GCRLB	2D	×	×	单目标	×	√	×
Li等人^[59]	MN-DDPG	2D	×	√	单目标	×	√	×
王祝等人^[60]	SCP	3D	×	×	多目标	√	√	√
严飞等人^[40]	协同PSO	2D	√	×	多目标	×	×	√
Zhang等人^[61]	模糊蚁群	2D	×	×	多目标	×	×	×
√：考虑；×：不考虑

下载: 导出CSV

表 6 无人机集群协同对抗方法研究对比

参考文献	方法	空间维度	异构集群	非完全信息	目标分配	协同态势评估	意图推断	协同优势
Ren等人^[63]	MADDPG	3D	×	√	×	√	√	√
Zhang等人^[64]	Actor-Critic	3D	×	×	√	√	×	√
Li等人^[65]	Actor-Critic	3D	×	×	×	≈	×	√
Wang等人^[66]	PD-MADDPG	2D	×	×	×	≈	×	√
Yu等人^[67]	CLPIO	3D	×	×	√	×	×	√
Deng等人^[68]	分布式任务分配	2D	√	√	√	×	√	√
√：考虑；×：不考虑；≈：采用了与协同态势评估类似的方法

下载: 导出CSV

表 7 无人机集群追踪决策方法研究对比

参考文献	方法	仿真维度	异构集群	目标数量	目标不确定运动	避障研究	复杂环境
Memon等人^[69]	sLM-IPDA	3D	×	多目标	√	×	√
Cybulski等人^[70]	多重有向图	2D	×	多目标	√	×	√
Yu等人^[71]	自适应差分进化-Nash	3D	×	单目标	√	×	√
Zhou等人^[72]	最大互惠奖励MARL	2D	×	多目标	√	×	√
Zhou等人^[73]	D3QN	2D	×	多目标	×	×	×
Hua等人^[74]	策略梯度 - 注意力	2D	×	单目标	×	√	√
√: 考虑；×: 不考虑

下载: 导出CSV

表 8 无人机集群合围决策相关研究对比

参考文献	方法	仿真维度	异构集群	复杂环境适应性	有限感知	目标数量	目标状态
张岱峰等人^[75]	狼群交互动力学模型	3D	√	√	√	多目标	移动
Ma等人^[76]	DO-NS	2D	×	×	×	单目标	移动
Murat Ozbek等人^[77]	改进DDPG	2D	×	×	×	单目标	移动
Li等人^[78]	改进DDPG	3D	×	√	×	单目标	移动
过劲劲等人^[79]	一致性协议	2D	×	×	√	多目标	移动
√: 考虑；×: 不考虑

下载: 导出CSV

参考文献(89)

[1]	ZUO Zongyu, LIU Cunjia, HAN Qinglong, et al. Unmanned aerial vehicles: Control methods and future challenges[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9(4): 601–614. doi: 10.1109/JAS.2022.105410.
[2]	AYAMGA M, AKABA S, and NYAABA A A. Multifaceted applicability of drones: A review[J]. Technological Forecasting and Social Change, 2021, 167: 120677. doi: 10.1016/j.techfore.2021.120677.
[3]	HASSANALIAN M and ABDELKEFI A. Classifications, applications, and design challenges of drones: A review[J]. Progress in Aerospace Sciences, 2017, 91: 99–131. doi: 10.1016/j.paerosci.2017.04.003.
[4]	ELMESEIRY N, ALSHAER N, and ISMAIL T. A detailed survey and future directions of Unmanned Aerial Vehicles (UAVs) with potential applications[J]. Aerospace, 2021, 8(12): 363. doi: 10.3390/aerospace8120363.
[5]	CLARKE R A and KNAKE R K. The Fifth Domain: Defending Our Country, Our Companies, and Ourselves in the Age of Cyber Threats[M]. London: Penguin Press, 2019: 58.
[6]	WU Ao, YANG Rennong, LIANG Xiaolong, et al. Visual range maneuver decision of unmanned combat aerial vehicle based on fuzzy reasoning[J]. International Journal of Fuzzy Systems, 2022, 24(1): 519–536. doi: 10.1007/s40815-021-01158-y.
[7]	ZHANG Lin, ZHU Yian, SHI Xianchen, et al. A situation assessment method with an improved fuzzy deep neural network for multiple UAVs[J]. Information, 2020, 11(4): 194. doi: 10.3390/INFO11040194.
[8]	HUANG Changqiang, DONG Kangsheng, HUANG Hanqiao, et al. Autonomous air combat maneuver decision using Bayesian inference and moving horizon optimization[J]. Journal of Systems Engineering and Electronics, 2018, 29(1): 86–97. doi: 10.21629/JSEE.2018.01.09.
[9]	XIE Lei, DING Dali, WEI Zhenglei, et al. Moving time UCAV maneuver decision based on the dynamic relational weight algorithm and trajectory prediction[J]. Mathematical Problems in Engineering, 2021, 2021: 6641567. doi: 10.1155/2021/6641567.
[10]	高杨, 李东生, 程泽新. 无人机分布式集群态势感知模型研究[J]. 电子与信息学报, 2018, 40(6): 1271–1278. doi: 10.11999/JEIT170877. GAO Yang, LI Dongsheng, and CHENG Zexin. UAV distributed swarm situation awareness model[J]. Journal of Electronics & Information Technology, 2018, 40(6): 1271–1278. doi: 10.11999/JEIT170877.
[11]	ENDSLEY M R. Toward a theory of situation awareness in dynamic systems[J]. Human Factors:The Journal of the Human Factors and Ergonomics Society, 1995, 37(1): 32–64. doi: 10.1518/001872095779049543.
[12]	SHIN H, LEE J, KIM H, et al. An autonomous aerial combat framework for two-on-two engagements based on basic fighter maneuvers[J]. Aerospace Science and Technology, 2018, 72: 305–315. doi: 10.1016/j.ast.2017.11.014.
[13]	高杨, 李东生, 柳向. 无人机集群协同态势觉察一致性评估[J]. 电子学报, 2019, 47(1): 1906–19. doi: 10.3969/j.issn.0372-2112.2019.01.025. GAO Yang, LI Dong-sheng, LIU Xiang. UAV Swarm Cooperative Situation Perception Consensus Evaluation[J]. Acta Electonica Sinica, 2019, 47(1): 1906–19. doi: 10.3969/j.issn.0372-2112.2019.01.025.
[14]	唐帅文, 周志杰, 姜江, 等. 考虑扰动的无人机集群协同态势感知一致性评估[J]. 航空学报, 2020, 41(S2): 724233. doi: 10.7527/S1000-6893.2020.24233. TANG Shuaiwen, ZHOU Zhijie, JIANG Jiang, et al. Consensus evaluation of UAV swarm cooperative situation awareness considering perturbation[J]. Acta Aeronauticaet Astronautica Sinica, 2020, 41(S2): 724233. doi: 10.7527/S1000-6893.2020.24233.
[15]	GAO Yang and LI Dongsheng. Unmanned aerial vehicle swarm distributed cooperation method based on situation awareness consensus and its information processing mechanism[J]. Knowledge-Based Systems, 2020, 188: 105034. doi: 10.1016/j.knosys.2019.105034.
[16]	韩博文, 姚佩阳, 钟赟, 等. 基于QABC-IFMADM算法的有人/无人机编队作战威胁评估[J]. 电子学报, 2018, 46(7): 1584–1592. doi: 10.3969/j.issn.0372-2112.2018.07.007. HAN Bowen, YAO Peiyang, ZHONG Yun, et al. Threat assessment of manned/unmanned aerial vehicle formation based on QABC-IFMADM algorithm[J]. Acta Electronica Sinica, 2018, 46(7): 1584–1592. doi: 10.3969/j.issn.0372-2112.2018.07.007.
[17]	ZHANG Kun, KONG Weiren, LIU Peipei, et al. Assessment and sequencing of air target threat based on intuitionistic fuzzy entropy and dynamic VIKOR[J]. Journal of Systems Engineering and Electronics, 2018, 29(2): 305–310. doi: 10.21629/JSEE.2018.02.11.
[18]	高杨, 黄仰超, 程国兵, 等. 直觉模糊信息下基于VIKOR和三支决策的多目标威胁评估方法[J]. 电子学报, 2021, 49(3): 542–549. doi: 10.12263/DZXB.20190150. GAO Yang, HUANG Yangchao, CHENG Guobing, et al. Multi-target threat assessment method based on VIKOR and three-way decisions under intuitionistic fuzzy information[J]. Acta Electronica Sinica, 2021, 49(3): 542–549. doi: 10.12263/DZXB.20190150.
[19]	GAO Yang, LI Dongsheng, and ZHONG Hua. A novel target threat assessment method based on three-way decisions under intuitionistic fuzzy multi-attribute decision making environment[J]. Engineering Applications of Artificial Intelligence, 2020, 87: 103276. doi: 10.1016/j.engappai.2019.103276.
[20]	ZHAO Ruojing, YANG Fengbao, JI Linna, et al. Dynamic air target threat assessment based on interval-valued intuitionistic fuzzy sets, game theory, and evidential reasoning methodology[J]. Mathematical Problems in Engineering, 2021, 2021: 6652706. doi: 10.1155/2021/6652706.
[21]	ZHAO Ruojing, YANG Fengbao, and JI Linna. An extended fuzzy CPT-TODIM model based on possibility theory and its application to air target dynamic threat assessment[J]. IEEE Access, 2022, 10: 21655–21669. doi: 10.1109/ACCESS.2022.3153361.
[22]	GAO Yang and LI Dongsheng. UAV swarm cooperative situation perception consensus evaluation method based on three-parameter interval number and heronian mean operator[J]. IEEE Access, 2018, 6: 73328–73340. doi: 10.1109/ACCESS.2018.2882409.
[23]	孟光磊, 周铭哲, 朴海音, 等. 基于协同战术识别的双机编队威胁评估方法[J]. 系统工程与电子技术, 2020, 42(10): 2285–2293. doi: 10.3969/j.issn.1001-506X.2020.10.17. MENG Guanglei, ZHOU Mingzhe, PIAO Haiyin, et al. Threat assessment method of dual-aircraft formation based on cooperative tactical recognition[J]. Systems Engineering and Electronics, 2020, 42(10): 2285–2293. doi: 10.3969/j.issn.1001-506X.2020.10.17.
[24]	GAO Yang and LI Dongsheng. Consensus evaluation method of multi-ground-target threat for unmanned aerial vehicle swarm based on heterogeneous group decision making[J]. Computers & Electrical Engineering, 2019, 74: 223–232. doi: 10.1016/j.compeleceng.2019.01.019.
[25]	LIU Chang, SUN Shaoshan, TAO Chenggang, et al. Sliding mode control of multi-agent system with application to UAV air combat[J]. Computers & Electrical Engineering, 2021, 96: 107491. doi: 10.1016/j.compeleceng.2021.107491.
[26]	PAN Qian, ZHOU Deyun, HUANG Jichuan, et al. Maneuver decision for cooperative close-range air combat based on state predicted influence diagram[C]. 2017 IEEE International Conference on Information and Automation (ICIA), Macao, China, 2017: 726–731. doi: 10.1109/ICInfA.2017.8079001.
[27]	XI Zhifei, XU An, KOU Yingxin, et al. Target maneuver trajectory prediction based on RBF neural network optimized by hybrid algorithm[J]. Journal of Systems Engineering and Electronics, 2021, 32(2): 498–516. doi: 10.23919/JSEE.2021.000042.
[28]	LIU D, ZONG Q, ZHANG X, et al. Game of drones: Intelligent online decision making of Multi-UAV confrontation[J]. IEEE Transactions on Emerging Topics in Computational Intelligence, 2024, 8(2): 2086–210. doi: 10.1109/TETCI.2024.3360282.
[29]	TAN Mulai, TANG Andi, DING Dali, et al. Autonomous air combat maneuvering decision method of UCAV based on LSHADE-TSO-MPC under enemy trajectory prediction[J]. Electronics, 2022, 11(20): 3383. doi: 10.3390/electronics11203383.
[30]	YANG Zhen, SUN Zhixiao, PIAO Haiyin, et al. An autonomous attack guidance method with high aiming precision for UCAV based on adaptive fuzzy control under model predictive control framework[J]. Applied Sciences, 2020, 10(16): 5677. doi: 10.3390/app10165677.
[31]	ZHOU Tongle, CHEN Mou, WANG Yuhui, et al. Information entropy-based intention prediction of aerial targets under uncertain and incomplete information[J]. Entropy, 2020, 22(3): 279. doi: 10.3390/e22030279.
[32]	刘钻东, 陈谋, 吴庆宪, 等. 非完备信息下无人机空战目标意图预测[J]. 中国科学:信息科学, 2020, 50(5): 704–717. doi: 10.1360/SSI-2019-0106. LIU Zuandong, CHEN Mou, WU Qingxian, et al. Prediction of unmanned aerial vehicle target intention under incomplete information[J]. Scientia Sinica Informationis, 2020, 50(5): 704–717. doi: 10.1360/SSI-2019-0106.
[33]	WANG Xingyu, YANG Zhen, ZHAN Guang, et al. Tactical intention recognition method of air combat target based on BiLSTM network[C]. 2022 IEEE International Conference on Unmanned Systems (ICUS), Guangzhou, China, 2022: 63–67. doi: 10.1109/ICUS55513.2022.9986667.
[34]	XIA Jingyang, CHEN Mengqi, and FANG Weiguo. Air combat intention recognition with incomplete information based on decision tree and GRU network[J]. Entropy, 2023, 25(4): 671. doi: 10.3390/e25040671.
[35]	WANG Yinhan, WANG Jiang, FAN Shipeng, et al. Quick intention identification of an enemy aerial target through information classification processing[J]. Aerospace Science and Technology, 2023, 132: 108005. doi: 10.1016/j.ast.2022.108005.
[36]	MENG Haodong, SUN Chong, FENG Yunchong, et al. One-to-one close air combat maneuver decision method based on target maneuver intention prediction[C]. 2022 IEEE International Conference on Unmanned Systems (ICUS), Guangzhou, China, 2022: 1454–1465. doi: 10.1109/ICUS55513.2022.9987174.
[37]	MENG Guanglei, ZHAO Runnan, WANG Biao, et al. Target tactical intention recognition in multiaircraft cooperative air combat[J]. International Journal of Aerospace Engineering, 2021, 2021: 9558838. doi: 10.1155/2021/9558838.
[38]	马钰棠, 孙鹏, 张杰勇, 等. 样本不平衡下的空中群组意图识别方法[J]. 系统工程与电子技术, 2022, 44(12): 3747–3755. doi: 10.12305/j.issn.1001-506X.2022.12.19. MA Yutang, SUN Peng, ZHANG Jieyong, et al. Air group intention recognition method under imbalance samples[J]. Systems Engineering and Electronics, 2022, 44(12): 3747–3755. doi: 10.12305/j.issn.1001-506X.2022.12.19.
[39]	WANG Siyuan, WANG Gang, FU Qiang, et al. STABC-IR: An air target intention recognition method based on bidirectional gated recurrent unit and conditional random field with space-time attention mechanism[J]. Chinese Journal of Aeronautics, 2023, 36(3): 316–334. doi: 10.1016/j.cja.2022.11.018.
[40]	严飞, 祝小平, 周洲, 等. 考虑同时攻击约束的多异构无人机实时任务分配[J]. 中国科学:信息科学, 2019, 49(5): 555–569. doi: 10.1360/N112018-00338. YAN Fei, ZHU Xiaoping, ZHOU Zhou, et al. Real-time task allocation for a heterogeneous multi-UAV simultaneous attack[J]. Scientia Sinica Informationis, 2019, 49(5): 555–569. doi: 10.1360/N112018-00338.
[41]	ZHEN Ziyang, WEN Liangdong, WANG Bolan, et al. Improved contract network protocol algorithm based cooperative target allocation of heterogeneous UAV swarm[J]. Aerospace Science and Technology, 2021, 119: 107054. doi: 10.1016/j.ast.2021.107054.
[42]	王峰, 黄子路, 韩孟臣, 等. 基于KnCMPSO算法的异构无人机协同多任务分配[J]. 自动化学报, 2023, 49(2): 399–414. doi: 10.16383/j.aas.c210696. WANG Feng, HUANG Zilu, HAN Mengchen, et al. A knee point based coevolution multi-objective particle swarm optimization algorithm for heterogeneous UAV cooperative multi-task allocation[J]. Acta Automatica Sinica, 2023, 49(2): 399–414. doi: 10.16383/j.aas.c210696.
[43]	ZHAO Xinyi, ZONG Quan, TIAN Bailing, et al. Fast task allocation for heterogeneous unmanned aerial vehicles through reinforcement learning[J]. Aerospace Science and Technology, 2019, 92: 588–594. doi: 10.1016/j.ast.2019.06.024.
[44]	JIA Zhenyue, YU Jianqiao, AI Xiaolin, et al. Cooperative multiple task assignment problem with stochastic velocities and time windows for heterogeneous unmanned aerial vehicles using a genetic algorithm[J]. Aerospace Science and Technology, 2018, 76: 112–125. doi: 10.1016/j.ast.2018.01.025.
[45]	赵玉亮, 宋业新, 张建军, 等. 基于多策略融合粒子群的无人机对地攻击模糊博弈决策[J]. 控制理论与应用, 2019, 36(10): 1644–1652. doi: 10.7641/CTA.2019.80437. ZHAO Yuliang, SONG Yexin, ZHANG Jianjun, et al. Fuzzy game decision-making of unmanned aerial vehicles air-to-ground attack based on the particle swarm optimization integrating multiply strategies[J]. Control Theory Applications, 2019, 36(10): 1644–1652. doi: 10.7641/CTA.2019.80437.
[46]	张安, 杨咪, 毕文豪, 等. 基于多策略GWO算法的不确定环境下异构多无人机任务分配[J]. 航空学报, 2023, 44(8): 327115. doi: 10.7527/S1000-6893.2022.2711. ZHANG An, YANG Mi, BI Wenhao, et al. Task allocation of heterogeneous multi-UAVs in uncertain environment based on multi-strategy integrated GWO[J]. Acta Aeronauticaet Astronautica Sinica, 2023, 44(8): 327115. doi: 10.7527/S1000-6893.2022.2711.
[47]	LIU Da, DOU Liquan, ZHANG Ruilong, et al. Multi-agent reinforcement learning-based coordinated dynamic task allocation for heterogenous UAVs[J]. IEEE Transactions on Vehicular Technology, 2023, 72(4): 4372–4383. doi: 10.1109/TVT.2022.3228198.
[48]	SUN Jiayi, TANG Jun, and LAO Songyang. Collision avoidance for cooperative UAVs with optimized artificial potential field algorithm[J]. IEEE Access, 2017, 5: 18382–18390. doi: 10.1109/ACCESS.2017.2746752.
[49]	HEIDARI H and SASKA M. Collision-free trajectory planning of multi-rotor UAVs in a wind condition based on modified potential field[J]. Mechanism and Machine Theory, 2021, 156: 104140. doi: 10.1016/j.mechmachtheory.2020.104140.
[50]	WU Enming, SUN Yidong, HUANG Jianyu, et al. Multi UAV cluster control method based on virtual core in improved artificial potential field[J]. IEEE Access, 2020, 8: 131647–131661. doi: 10.1109/ACCESS.2020.3009972.
[51]	SELVAM P K, RAJA G, RAJAGOPAL V, et al. Collision-free path planning for UAVs using efficient artificial potential field algorithm[C]. 2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring), Helsinki, Finland, 2021: 1–5. doi: 10.1109/VTC2021-Spring51267.2021.9448937.
[52]	PAN Zhenhua, ZHANG Chengxi, XIA Yuanqing, et al. An improved artificial potential field method for path planning and formation control of the multi-UAV systems[J]. IEEE Transactions on Circuits and Systems II:Express Briefs, 2022, 69(3): 1129–1133. doi: 10.1109/TCSII.2021.3112787.
[53]	SINGLA A, PADAKANDLA S, and BHATNAGAR S. Memory-based deep reinforcement learning for obstacle avoidance in UAV with limited environment knowledge[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(1): 107–118. doi: 10.1109/TITS.2019.2954952.
[54]	申燕凯, 段海滨, 邓亦敏, 等. 仿鸽群被动式惯性应急避障的无人机集群飞行验证[J]. 中国科学: 信息科学, 2019, 49(10): 1343–1352. doi: 10.1360/N112018-00333. SHEN Yankai, DUAN Haibin, DENG Yimin, et al. Verification of a UAV swarm flight simulating the passive inertial emergency obstacle avoidance behavior of a pigeon flock[J]. Scientia Sinica Informationis, 2019, 49: 1343–1352. doi: 10.1360/N112018-00333.
[55]	姜龙亭, 魏瑞轩, 张启瑞, 等. 基于群智机理的集群防碰撞控制[J]. 航空学报, 2020, 41(S2): 724294. doi: 10.7527/S1000-6893.2020.24294. JIANG Longting, WEI Ruixuan, ZHANG Qirui, et al. Anti-collision control of UAVs based on swarm intelligence mechanism[J]. Acta Aeronauticaet Astronautica Sinica, 2020, 41(S2): 724294. doi: 10.7527/S1000-6893.2020.24294.
[56]	WU Yu, GOU Jinzhan, HU Xinting, et al. A new consensus theory-based method for formation control and obstacle avoidance of UAVs[J]. Aerospace Science and Technology, 2020, 107: 106332. doi: 10.1016/j.ast.2020.106332.
[57]	蔡星娟, 胡钊鸣, 张志霞, 等. 基于高维多目标优化的多无人机协同航迹规划[J]. 中国科学:信息科学, 2021, 51(6): 985–996. doi: 10.1360/SSI-2020-0218. CAI Xingjuan, HU Zhaoming, ZHANG Zhixia, et al. Multi-UAV coordinated path planning based on many-objective optimization[J]. Scientia Sinica Informationis, 2021, 51(6): 985–996. doi: 10.1360/SSI-2020-0218.
[58]	左燕, 刘雪娇, 彭冬亮. 距离相关噪声AOA协同定位下无人机路径优化方法[J]. 电子与信息学报, 2021, 43(4): 1192–1198. doi: 10.11999/JEIT200078. ZUO Yan, LIU Xuejiao, and PENG Dongliang. UAV path planning for AOA-based source localization with distance-dependent noises[J]. Journal of Electronics & Information Technology, 2021, 43(4): 1192–1198. doi: 10.11999/JEIT200078.
[59]	LI Bo, YANG Zhipeng, CHEN Daqing, et al. Maneuvering target tracking of UAV based on MN-DDPG and transfer learning[J]. Defence Technology, 2021, 17(2): 457–466. doi: 10.1016/j.dt.2020.11.014.
[60]	王祝, 徐广通, 龙腾. 基于定制内点法的多无人机协同轨迹规划[J/OL]. 自动化学报, 1–12. https://kns.cnki.net/kns8/Detail?sfield=fn&QueryID=0&CurRec=1&recid=&FileName=MOTO20201119000&DbName=CAPJLAST&DbCode=CAPJ&yx=Y&pr=&URLID=11.2109.TP.20201119.1423.001, 2020. WANG Zhu, XU Guangtong, and LONG Teng. Customized interior-point method for cooperative trajectory planning of unmanned aerial vehicles[J/OL]. Acta Automatica Sinica, 1–12. https://kns.cnki.net/kns8/Detail?sfield=fn&QueryID=0&CurRec=1&recid=&FileName=MOTO20201119000&DbName=CAPJLAST&DbCode=CAPJ&yx=Y&pr=&URLID=11.2109.TP.20201119.1423.001, 2020.
[61]	ZHANG Lin, ZHU Yian, and SHI Xianchen. A hierarchical decision-making method with a fuzzy ant colony algorithm for mission planning of multiple UAVs[J]. Information, 2020, 11(4): 226. doi: 10.3390/INFO11040226.
[62]	LI Zhenyu, ZHU Hai, and LUO Yazhong. An escape strategy in orbital pursuit-evasion games with incomplete information[J]. Science China Technological Sciences, 2021, 64(3): 559–570. doi: 10.1007/s11431-020-1662-0.
[63]	REN Zhi, ZHANG Dong, TANG Shuo, et al. Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game[J]. Defence Technology, 2023, 27: 308–317. doi: 10.1016/j.dt.2022.10.008.
[64]	ZHANG Jiandong, YANG Qiming, SHI Guoqing, et al. UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning[J]. Journal of Systems Engineering and Electronics, 2021, 32(6): 1421–1438. doi: 10.23919/JSEE.2021.000121.
[65]	LI Shaowei, JIA Yuhong, YANG Fan, et al. Collaborative decision-making method for multi-UAV based on multiagent reinforcement learning[J]. IEEE Access, 2022, 10: 91385–91396. doi: 10.1109/ACCESS.2022.3199070.
[66]	WANG Zhenhua, GUO Yan, LI Ning, et al. Autonomous collaborative combat strategy of unmanned system group in continuous dynamic environment based on PD-MADDPG[J]. Computer Communications, 2023, 200: 182–204. doi: 10.1016/j.comcom.2023.01.009.
[67]	YU Yueping, LIU Jichuan, and WEI Chen. Hawk and pigeon’s intelligence for UAV swarm dynamic combat game via competitive learning pigeon-inspired optimization[J]. Science China Technological Sciences, 2022, 65(5): 1072–1086. doi: 10.1007/s11431-021-1951-9.
[68]	DENG Hanqiang, HUANG Jian, LIU Quan, et al. A distributed collaborative allocation method of reconnaissance and strike tasks for heterogeneous UAVs[J]. Drones, 2023, 7(2): 138. doi: 10.3390/drones7020138.
[69]	MEMON S A and ULLAH I. Detection and tracking of the trajectories of dynamic UAVs in restricted and cluttered environment[J]. Expert Systems with Applications, 2021, 183: 115309. doi: 10.1016/j.eswa.2021.115309.
[70]	CYBULSKI P and ZIELIŃSKI Z. UAV swarms behavior modeling using tracking bigraphical reactive systems[J]. Sensors, 2021, 21(2): 622. doi: 10.3390/s21020622.
[71]	YU Yao, WANG Hongli, LIU Shumei, et al. Distributed multi-agent target tracking: A Nash-combined adaptive differential evolution method for UAV systems[J]. IEEE Transactions on Vehicular Technology, 2021, 70(8): 8122–8133. doi: 10.1109/TVT.2021.3091575.
[72]	ZHOU Wenhong, LI Jie, LIU Zhihong, et al. Improving multi-target cooperative tracking guidance for UAV swarms using multi-agent reinforcement learning[J]. Chinese Journal of Aeronautics, 2022, 35(7): 100–112. doi: 10.1016/j.cja.2021.09.008.
[73]	ZHOU Wenhong, LIU Zhihong, LI Jie, et al. Multi-target tracking for unmanned aerial vehicle swarms using deep reinforcement learning[J]. Neurocomputing, 2021, 466: 285–297. doi: 10.1016/j.neucom.2021.09.044.
[74]	HUA Xia, WANG Xinqing, RUI Ting, et al. Light-weight UAV object tracking network based on strategy gradient and attention mechanism[J]. Knowledge-Based Systems, 2021, 224: 107071. doi: 10.1016/j.knosys.2021.107071.
[75]	张岱峰, 段海滨, 范彦铭. 仿狼群狩猎空间交互机制的无人机集群合围控制[J]. 中国科学:技术科学, 2022, 52(10): 1555–1570. doi: 10.1360/SST-2021-0042. ZHANG Daifeng, DUAN Haibin, and FAN Yanming. UAV swarm containment control inspired by spatial interaction mechanism of wolf-pack foraging[J]. Scientia Sinica Technologica, 2022, 52(10): 1555–1570. doi: 10.1360/SST-2021-0042.
[76]	MA Yingying, WANG Guoqiang, HU Xiaoxuan, et al. Cooperative occupancy decision making of multi-UAV in beyond-visual-range air combat: A game theory approach[J]. IEEE Access, 2020, 8: 11624–11634. doi: 10.1109/ACCESS.2019.2933022.
[77]	MURAT OZBEK M and KOYUNCU E. Reinforcement learning based air combat maneuver generation[J]. arXiv: 2201.05528, 2022.
[78]	LI Yue, HAN Wei, and WANG Yongqing. Deep reinforcement learning with application to air confrontation intelligent decision-making of manned/unmanned aerial vehicle cooperative system[J]. IEEE Access, 2020, 8: 67887–67898. doi: 10.1109/ACCESS.2020.2985576.
[79]	过劲劲, 齐俊桐, 王明明, 等. 未知区域中四旋翼无人机集群协同搜索与围捕算法[J]. 北京航空航天大学学报, 2023, 49(8): 2001–2010. doi: 10.13700/j.bh.1001-5965.2021.0606. GUO Jinjin, QI Juntong, WANG Mingming, et al. A cooperative search and encirclement algorithm for quadrotors in unknown areas[J]. Journal of Beijing University of Aeronautics and Astronautics, 2023, 49(8): 2001–2010. doi: 10.13700/j.bh.1001-5965.2021.0606.
[80]	牛轶峰, 沈林成, 李杰, 等. 无人-有人机协同控制关键问题[J]. 中国科学:信息科学, 2019, 49(5): 538–554. doi: 10.1360/N112019-00008. NIU Yifeng, SHEN Lincheng, LI Jie, et al. Key scientific problems in cooperation control of unmanned-manned aircraft systems[J]. Scientia Sinica Informationis, 2019, 49(5): 538–554. doi: 10.1360/N112019-00008.
[81]	KHAWAJA W, SEMKIN V, RATYAL N I, et al. Threats from and countermeasures for unmanned aerial and underwater vehicles[J]. Sensors, 2022, 22(10): 3896. doi: 10.3390/s22103896.
[82]	YANG Xuekuan, WANG Wei, and HUANG Ping. Distributed optimal consensus with obstacle avoidance algorithm of mixed-order UAVs–USVs–UUVs systems[J]. ISA Transactions, 2020, 107: 270–286. doi: 10.1016/j.isatra.2020.07.028.
[83]	XUE Kai and WU Tingyi. Distributed consensus of USVs under heterogeneous UAV-USV multi-agent systems cooperative control scheme[J]. Journal of Marine Science and Engineering, 2021, 9(11): 1314. doi: 10.3390/jmse 9111314.
[84]	WEI Wei, WANG Jingjing, FANG Zhengru, et al. 3U: Joint design of UAV-USV-UUV networks for cooperative target hunting[J]. IEEE Transactions on Vehicular Technology, 2023, 72(3): 4085–4090. doi: 10.1109/TVT.2022.3220856.
[85]	XU Qingzheng, WANG Na, WANG Lei, et al. Multi-task optimization and multi-task evolutionary computation in the past five years: A brief review[J]. Mathematics, 2021, 9(8): 864. doi: 10.3390/math9080864.
[86]	OSABA E, DEL SER J, MARTINEZ A D, et al. Evolutionary multitask optimization: A methodological overview, challenges, and future research directions[J]. Cognitive Computation, 2022, 14(3): 927–954. doi: 10.1007/s12559-022-10012-8.
[87]	WONG Annie, BÄCK T, KONONOVA A V, et al. Deep multiagent reinforcement learning: Challenges and directions[J]. Artificial Intelligence Review, 2023, 56(6): 5023–5056. doi: 10.1007/s10462-022-10299-x.
[88]	OROOJLOOY A and HAJINEZHAD D. A review of cooperative multi-agent deep reinforcement learning[J]. Applied Intelligence, 2023, 53(11): 13677–13722. doi: 10.1007/s10489-022-04105-y.
[89]	CANESE L, CARDARILLI G C, DI NUNZIO L, et al. Multi-agent reinforcement learning: A review of challenges and applications[J]. Applied Sciences, 2021, 11(11): 4948. doi: 10.3390/app11114948.