A General Evaluation Framework for Mission Planning Algorithms for Remote Sensing Satellite Constellations
-
摘要: 当前卫星遥感已成为国土资源普查和防灾减灾的关键工具,随着卫星制造和发射能力不断增强,卫星数量规模快速扩展,遥感星群任务规划显得尤为重要,星群任务规划算法面临从单星静态调度到星群动态协同的范式跃迁。星群任务规划需要根据各颗卫星轨道分布、载荷约束条件以及任务要求,为每颗卫星制定观测计划,实现星群整体观测效益最大化。然而当前学术界对遥感卫星星群任务规划算法测评尚未形成统一的量化标准,不同研究涉及的问题规模和场景跨度巨大,这使得跨论文的性能比较几乎不可能。本文提出面向遥感卫星星群任务规划算法的通用测评框架RSCMP-Bench,主要贡献体现为3个方面: 多场景标准任务库、多维效能评估指标体系、仿真与评测平台。通过开展基准测试,验证了RSCMP-Bench测试流程的可行性、可复现性及其对算法性能评估的区分能力。测评框架及运行环境已发布在天智杯在线平台,RSCMP-Bench旨在为领域建立类似大规模标注图像数据集(ImageNet)之于计算机视觉、通用语言理解评测集(GLUE)之于自然语言处理的统一基准,推动卫星任务规划算法从“孤立实验”走向“标准化竞技”,最终加速面向下一代智能遥感星群的规划技术突破。Abstract:
Objective The rapid growth in remote sensing satellite constellations has shifted mission planning from single-satellite static scheduling to large-scale dynamic coordination across heterogeneous constellations. However, evaluation methods have not kept pace with algorithm development. Existing studies often rely on private datasets, simplified metrics centered on Completion Rate, and idealized simulations that ignore realistic constraints, such as attitude maneuvers, illumination conditions, and dynamic task insertion. These limitations prevent fair cross-paper comparison and slow engineering application. To address this gap, this paper proposes the Remote Sensing Constellation Mission Planning Benchmark (RSCMP-Bench), a general, open, and reproducible evaluation framework. It is designed as a unified benchmark for the community, similar to ImageNet in computer vision and General Language Understanding Evaluation (GLUE) in Natural Language Processing (NLP). Methods RSCMP-Bench consists of three components. First, the multi-scenario standard task library contains 300 standardized scenarios at three difficulty levels: Low, Medium, and High, with 100 scenarios per level. Satellite numbers range from 30 to 200, and task demands range from 56 to 560. All scenarios are generated from public Two-Line Element (TLE) data and explicitly model realistic constraints. Optical satellites require a minimum solar elevation angle, and Synthetic Aperture Radar (SAR) satellites require incidence angles within specified ranges. General constraints, such as per-orbit maximum on-time, minimum single-operation on-time, attitude maneuver time, and valid execution windows, are also modeled. The scenarios include point tasks, area tasks, static tasks, and dynamically inserted tasks. Second, the multi-dimensional effectiveness evaluation system includes a Basic Performance layer and a Dynamic Adaptability layer. The Basic Performance layer uses Completion Rate, Weighted Completion Rate, Average Response Delay, and Time Utilization. The Dynamic Adaptability layer uses multi-stage rolling evaluation with random dynamic task insertion. The Dynamic Adaptability Score measures the post-insertion Completion Rate relative to the baseline, and Dynamic Response Efficiency measures the performance gain per unit replanning time. A composite RSCMP-Bench Score is also provided. Third, the simulation and evaluation platform uses a client-server architecture. It integrates a Simplified General Perturbations 4 (SGP4) propagator, algorithm adapters, two-stage constraint verification, an intelligent scenario generator, and visualization tools. The platform has been deployed at https://www.tianzhibei.com and has supported a national competition with more than 80 research teams. Results and Discussions Baseline experiments comparing Random Scheduler and Priority Greedy validate the feasibility, reproducibility, and discriminative capacity of RSCMP-Bench. Random Scheduler yields very low Completion Rates of 7.3%, 3.8%, and 1.9% on the Low, Medium, and High levels, respectively. These results confirm the extreme sparsity of the feasible solution space. Priority Greedy achieves higher Completion Rates but still degrades as scenario difficulty increases, decreasing from 76.1% at the Low level to 63.7% at the Medium level and 49.2% at the High level. These findings indicate that high-difficulty scenarios remain challenging even for reasonable heuristic methods. They also show considerable room for more advanced algorithms. The dynamic adaptability protocol quantifies algorithm robustness under unexpected dynamic task insertion, which is not captured by static evaluations. The two-stage constraint verification module rejects infeasible plans and generates detailed error reports to support debugging. Conclusions RSCMP-Bench provides a unified, fair, and reproducible benchmark for remote sensing constellation mission planning. By combining a public library of 300 standardized scenarios, a multi-dimensional effectiveness evaluation system based on Basic Performance and Dynamic Adaptability, and a simulation and evaluation platform with realistic constraints and automated scenario generation, the framework addresses the long-standing lack of standardized evaluation in this field. Baseline results confirm its discriminative capacity and reveal clear performance bottlenecks in large-scale dynamic scenarios. Inspired by ImageNet and GLUE, RSCMP-Bench can support systematic community evaluation and fair competition. The framework has been deployed at https://www.tianzhibei.com, and its adoption can accelerate progress in intelligent mission planning for next-generation remote sensing constellations. -
表 1 任务场景设置
难度 资源
普查点资源普查
区域追加点 追加区域 光学卫星 SAR卫星 低 45 5 5 1 20 10 中 180 20 20 5 52 28 高 450 50 50 10 132 68 表 2 测评数据核心数据表及其字段说明
数据表 字段 类型 说明 卫星
星历geometry POINT 星下点经纬度(WGS84) time INTEGER 从UTC0点开始的秒数 altitude REAL 卫星高度(km) sun_elevation REAL 星下点太阳高度角(°) 点任务 mission_id TEXT 任务唯一标识符 geometry POINT 任务地理位置 priority INTEGER 优先级(1~10) frequency INTEGER 要求执行次数 min_interval REAL 最小观测间隔(h) time_start INTEGER 最早可执行时间(s) time_end INTEGER 最晚可执行时间(s) 区域
任务mission_id TEXT 任务唯一标识符 geometry POLYGON 区域边界 area REAL 区域面积
(km2, EPSG:6933 )coverage_ratio REAL 要求覆盖率 priority INTEGER 优先级 time_start INTEGER 最早可执行时间(s) time_end INTEGER 最晚可执行时间(s) 卫星 satellite_id TEXT 卫星标识符 type TEXT optical/SAR swath REAL 幅宽(km) max_roll_left REAL 最大左侧摆角(°) max_roll_right REAL 最大右侧摆角(°) min_sun_elev REAL 最小太阳高度角(光学) min_inc_angle REAL 最小入射角(SAR) max_inc_angle REAL 最大入射角(SAR) resolution REAL 分辨率(m) min_on_time REAL 单次最小开机时长(s) max_on_time_orbit REAL 单圈最大开机时长(s) trans_time_0 REAL 0°侧摆机动时长(s) trans_time_10 REAL 10°侧摆机动时长(s) trans_time_20 REAL 20°侧摆机动时长(s) 表 3 基线模型在不同难度场景下的任务完成率(%)
模型 低难度 (L1) 中难度 (L2) 高难度 (L3) 随机调度 7.3 3.8 1.9 优先级贪婪算法 76.1 63.7 49.2 -
[1] 江碧涛. 我国空间对地观测技术的发展与展望[J]. 测绘学报, 2022, 51(7): 1153–1159. doi: 10.11947/j.AGCS.2022.20220199.JIANG Bitao. The development and prospect of China's space earth observation technology[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7): 1153–1159. doi: 10.11947/j.AGCS.2022.20220199. [2] 康利鸿, 田菁, 江碧涛. 巨星座时代遥感卫星应用技术挑战与思考[J]. 遥感学报, 2024, 28(7): 1658–1666. doi: 10.11834/jrs.20233248.KANG Lihong, TIAN Jing, and JIANG Bitao. Challenges and research on remote sensing satellite application technology in the Giant Constellation Era[J]. National Remote Sensing Bulletin, 2024, 28(7): 1658–1666. doi: 10.11834/jrs.20233248. [3] DENG Jia, DONG Wei, SOCHER R, et al. ImageNet: A large-scale hierarchical image database[C]. 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, 2009: 248–255. doi: 10.1109/CVPR.2009.5206848. [4] WANG A, SINGH A, MICHAEL J, et al. GLUE: A multi-task benchmark and analysis platform for natural language understanding[C]. The 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Brussels, Belgium, 2018. doi: 10.18653/v1/W18-5446. [5] 天智杯人工智能挑战赛平台. 面向国土资源普查的智能任务规划算法比赛[EB/OL]. https://www.tianzhibei.com, 2024. Tianzhibei Online Platform. Intelligent mission planning algorithm competition for land resource surveying[EB/OL]. https://www.tianzhibei.com, 2024. [6] 杜永浩, 张本奎, 吴健, 等. 大规模遥感卫星智能任务调度方法研究进展[J]. 电子与信息学报, 2025, 47(12): 5033–5047. doi: 10.11999/JEIT251038.DU Yonghao, ZHANG Benkui, WU Jian, et al. Survey on intelligent methods for large-scale remote sensing satellite scheduling[J]. Journal of Electronics & Information Technology, 2025, 47(12): 5033–5047. doi: 10.11999/JEIT251038. [7] 李庥甜, 王凌, 陈英武, 等. 基于自适应大邻域搜索的多场景多卫星任务规划方法[J]. 系统仿真学报, 2025, 37(7): 1836–1847. doi: 10.16182/j.issn1004731x.joss.25-0095.LI Xiutian, WANG Ling, CHEN Yingwu, et al. Multi-scenario multi-satellite mission planning method based on adaptive large neighborhood search[J]. Journal of System Simulation, 2025, 37(7): 1836–1847. doi: 10.16182/j.issn1004731x.joss.25-0095. [8] SHE Yucheng, YANG Zhi, and WANG Dandan. A study on adaptive multi-satellite mission allocation algorithm for efficient large-scale constellation planning[J]. Journal of Physics: Conference Series, 2025, 3083(1): 012002. doi: 10.1088/1742-6596/3083/1/012002. [9] HUANG Weiquan, WANG He, YI Dongbo, et al. A multiple agile satellite staring observation mission planning method for dense regions[J]. Remote Sensing, 2023, 15(22): 5317. doi: 10.3390/rs15225317. [10] 秦嘉豪, 李宝卫, 白雪, 等. 面向异质卫星集群的事件触发分布式自主任务规划方法[J]. 中国空间科学技术(中英文), 2025, 45(4): 88–101. doi: 10.16708/j.cnki.1000-758X.2025.0061.QIN Jiahao, LI Baowei, BAI Xue, et al. Distributed autonomous scheduling based on event trigger for heterogeneous satellite swarm[J]. Chinese Space Science and Technology, 2025, 45(4): 88–101. doi: 10.16708/j.cnki.1000-758X.2025.0061. [11] HILTON S, THANGAVEL K, GARDI A, et al. Intelligent mission planning for autonomous distributed satellite systems[J]. Acta Astronautica, 2024, 225: 123–136. doi: 10.1016/j.actaastro.2024.08.050. [12] 袁健波, 杜永浩, 陈盈果, 等. 面向点群与大区域目标的成像卫星任务规划模型与算法研究[J]. 系统工程与电子技术, 2025, 47(9): 2939–2950. doi: 10.12305/j.issn.1001-506X.2025.09.15.YUAN Jianbo, DU Yonghao, CHEN Yingguo, et al. Research on imaging satellite mission planning model and algorithm for point-cluster and large-region targets[J]. Systems Engineering and Electronics, 2025, 47(9): 2939–2950. doi: 10.12305/j.issn.1001-506X.2025.09.15. [13] LONG Jun, WU Shimin, HAN Xiaodong, et al. Autonomous task planning method for multi-satellite system based on a hybrid genetic algorithm[J]. Aerospace, 2023, 10(1): 70. doi: 10.3390/aerospace10010070. [14] CHEN Yaxin, SHEN Xin, ZHANG Guo, et al. Large-scale multi-objective imaging satellite task planning algorithm for vast area mapping[J]. Remote Sensing, 2023, 15(17): 4178. doi: 10.3390/rs15174178. [15] 许可, 孙昌浩, 谢睿达, 等. 基于DQN的对地观测卫星调度算法[J]. 空间控制技术与应用(中英文), 2026, 52(1): 68–78. doi: 10.3969/j.issn.1674-1579.2026.01.007.XU Ke, SUN Changhao, XIE Ruida, et al. Earth observation satellite scheduling based on DQN[J]. Aerospace Control and Application, 2026, 52(1): 68–78. doi: 10.3969/j.issn.1674-1579.2026.01.007. [16] 陈盈果, 王斐然, 胡云鹏, 等. 融合大语言模型与强化学习的敏捷卫星任务分配算法设计[J]. 电子与信息学报, 2025, 47(12): 4959–4972. doi: 10.11999/JEIT250991.CHEN Yingguo, WANG Feiran, HU Yunpeng, et al. Automating algorithm design for agile satellite task assignment with large language models and reinforcement learning[J]. Journal of Electronics & Information Technology, 2025, 47(12): 4959–4972. doi: 10.11999/JEIT250991. [17] WU Xiande, ZHANG Zehua, CHEN Zhengnan, et al. Intelligent task planning method for distributed satellite system based on reinforcement learning[J]. Aerospace Science and Technology, 2026, 168: 111325. doi: 10.1016/j.ast.2025.111325. [18] HE Xiaohe, XIANG Junyan, YAN Mubiao, et al. Integrated clustering and mission planning for agile Earth observation satellite constellations[J]. IEICE Transactions on Electronics, 2025, E108. C(11): 594–597. doi: 10.1587/transele.2024ECS6016. [19] LI Shuo, WANG Gang, and CHEN Jinyong. AEM-D3QN: A graph-based deep reinforcement learning framework for dynamic earth observation satellite mission planning[J]. Aerospace, 2025, 12(5): 420. doi: 10.3390/aerospace12050420. [20] 杜永浩, 黎磊, 徐世龙, 等. 基于智能优化算法引擎的可演进星群智能任务规划[J]. 电子与信息学报, 2025, 47(6): 1645–1657. doi: 10.11999/JEIT240974.DU Yonghao, LI Lei, XU Shilong, et al. Evolutionary optimization for satellite constellation task scheduling based on intelligent optimization engine[J]. Journal of Electronics & Information Technology, 2025, 47(6): 1645–1657. doi: 10.11999/JEIT240974. [21] 魏普远, 何磊. 基于深度强化学习的自适应大邻域搜索算法在成像卫星调度问题中的应用[J]. 电子与信息学报, 2025, 47(12): 5005–5015. doi: 10.11999/JEIT251009.WEI Puyuan and HE Lei. A deep reinforcement learning enhanced adaptive large neighborhood search for imaging satellite scheduling[J]. Journal of Electronics & Information Technology, 2025, 47(12): 5005–5015. doi: 10.11999/JEIT251009. [22] SAMVELYAN M, RASHID T, DE WITT C S, et al. The StarCraft multi-agent challenge[C]. The 18th International Conference on Autonomous Agents and MultiAgent Systems, Montreal, Canada, 2019. -
下载: