A General Evaluation Framework for Mission Planning Algorithms for Remote Sensing Satellite Constellations

LI Jinfei; YU Xiaogang; TIAN Jing; HE Haochen; XING Xiangwei; ZHANG Xiaohan

doi:10.11999/JEIT260335

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

LI Jinfei, YU Xiaogang, TIAN Jing, HE Haochen, XING Xiangwei, ZHANG Xiaohan. A General Evaluation Framework for Mission Planning Algorithms for Remote Sensing Satellite Constellations[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260335

Citation:

LI Jinfei, YU Xiaogang, TIAN Jing, HE Haochen, XING Xiangwei, ZHANG Xiaohan. A General Evaluation Framework for Mission Planning Algorithms for Remote Sensing Satellite Constellations[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260335

Citation:

LI Jinfei, YU Xiaogang, TIAN Jing, HE Haochen, XING Xiangwei, ZHANG Xiaohan. A General Evaluation Framework for Mission Planning Algorithms for Remote Sensing Satellite Constellations[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260335

PDF( 1080 KB)

A General Evaluation Framework for Mission Planning Algorithms for Remote Sensing Satellite Constellations

doi: 10.11999/JEIT260335 cstr: 32379.14.JEIT260335

Institute of remote sensing information , Beijing100011, China

Received Date: 2026-03-25
Accepted Date: 2026-05-29
Rev Recd Date: 2026-05-24

Available Online: 2026-06-10

Abstract

Abstract

Objective The rapid growth in remote sensing satellite constellations has shifted mission planning from single-satellite static scheduling to large-scale dynamic coordination across heterogeneous constellations. However, evaluation methods have not kept pace with algorithm development. Existing studies often rely on private datasets, simplified metrics centered on Completion Rate, and idealized simulations that ignore realistic constraints, such as attitude maneuvers, illumination conditions, and dynamic task insertion. These limitations prevent fair cross-paper comparison and slow engineering application. To address this gap, this paper proposes the Remote Sensing Constellation Mission Planning Benchmark (RSCMP-Bench), a general, open, and reproducible evaluation framework. It is designed as a unified benchmark for the community, similar to ImageNet in computer vision and General Language Understanding Evaluation (GLUE) in Natural Language Processing (NLP). Methods RSCMP-Bench consists of three components. First, the multi-scenario standard task library contains 300 standardized scenarios at three difficulty levels: Low, Medium, and High, with 100 scenarios per level. Satellite numbers range from 30 to 200, and task demands range from 56 to 560. All scenarios are generated from public Two-Line Element (TLE) data and explicitly model realistic constraints. Optical satellites require a minimum solar elevation angle, and Synthetic Aperture Radar (SAR) satellites require incidence angles within specified ranges. General constraints, such as per-orbit maximum on-time, minimum single-operation on-time, attitude maneuver time, and valid execution windows, are also modeled. The scenarios include point tasks, area tasks, static tasks, and dynamically inserted tasks. Second, the multi-dimensional effectiveness evaluation system includes a Basic Performance layer and a Dynamic Adaptability layer. The Basic Performance layer uses Completion Rate, Weighted Completion Rate, Average Response Delay, and Time Utilization. The Dynamic Adaptability layer uses multi-stage rolling evaluation with random dynamic task insertion. The Dynamic Adaptability Score measures the post-insertion Completion Rate relative to the baseline, and Dynamic Response Efficiency measures the performance gain per unit replanning time. A composite RSCMP-Bench Score is also provided. Third, the simulation and evaluation platform uses a client-server architecture. It integrates a Simplified General Perturbations 4 (SGP4) propagator, algorithm adapters, two-stage constraint verification, an intelligent scenario generator, and visualization tools. The platform has been deployed at https://www.tianzhibei.com and has supported a national competition with more than 80 research teams. Results and Discussions Baseline experiments comparing Random Scheduler and Priority Greedy validate the feasibility, reproducibility, and discriminative capacity of RSCMP-Bench. Random Scheduler yields very low Completion Rates of 7.3%, 3.8%, and 1.9% on the Low, Medium, and High levels, respectively. These results confirm the extreme sparsity of the feasible solution space. Priority Greedy achieves higher Completion Rates but still degrades as scenario difficulty increases, decreasing from 76.1% at the Low level to 63.7% at the Medium level and 49.2% at the High level. These findings indicate that high-difficulty scenarios remain challenging even for reasonable heuristic methods. They also show considerable room for more advanced algorithms. The dynamic adaptability protocol quantifies algorithm robustness under unexpected dynamic task insertion, which is not captured by static evaluations. The two-stage constraint verification module rejects infeasible plans and generates detailed error reports to support debugging. Conclusions RSCMP-Bench provides a unified, fair, and reproducible benchmark for remote sensing constellation mission planning. By combining a public library of 300 standardized scenarios, a multi-dimensional effectiveness evaluation system based on Basic Performance and Dynamic Adaptability, and a simulation and evaluation platform with realistic constraints and automated scenario generation, the framework addresses the long-standing lack of standardized evaluation in this field. Baseline results confirm its discriminative capacity and reveal clear performance bottlenecks in large-scale dynamic scenarios. Inspired by ImageNet and GLUE, RSCMP-Bench can support systematic community evaluation and fair competition. The framework has been deployed at https://www.tianzhibei.com, and its adoption can accelerate progress in intelligent mission planning for next-generation remote sensing constellations.
- Remote sensing satellite constellation,
- Mission planning algorithm,
- Benchmark framework

FullText(HTML)

References(22)

References

[1]	江碧涛. 我国空间对地观测技术的发展与展望[J]. 测绘学报, 2022, 51(7): 1153–1159. doi: 10.11947/j.AGCS.2022.20220199. JIANG Bitao. The development and prospect of China's space earth observation technology[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7): 1153–1159. doi: 10.11947/j.AGCS.2022.20220199.
[2]	康利鸿, 田菁, 江碧涛. 巨星座时代遥感卫星应用技术挑战与思考[J]. 遥感学报, 2024, 28(7): 1658–1666. doi: 10.11834/jrs.20233248. KANG Lihong, TIAN Jing, and JIANG Bitao. Challenges and research on remote sensing satellite application technology in the Giant Constellation Era[J]. National Remote Sensing Bulletin, 2024, 28(7): 1658–1666. doi: 10.11834/jrs.20233248.
[3]	DENG Jia, DONG Wei, SOCHER R, et al. ImageNet: A large-scale hierarchical image database[C]. 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, 2009: 248–255. doi: 10.1109/CVPR.2009.5206848.
[4]	WANG A, SINGH A, MICHAEL J, et al. GLUE: A multi-task benchmark and analysis platform for natural language understanding[C]. The 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Brussels, Belgium, 2018. doi: 10.18653/v1/W18-5446.
[5]	天智杯人工智能挑战赛平台. 面向国土资源普查的智能任务规划算法比赛[EB/OL]. https://www.tianzhibei.com, 2024. Tianzhibei Online Platform. Intelligent mission planning algorithm competition for land resource surveying[EB/OL]. https://www.tianzhibei.com, 2024.
[6]	杜永浩, 张本奎, 吴健, 等. 大规模遥感卫星智能任务调度方法研究进展[J]. 电子与信息学报, 2025, 47(12): 5033–5047. doi: 10.11999/JEIT251038. DU Yonghao, ZHANG Benkui, WU Jian, et al. Survey on intelligent methods for large-scale remote sensing satellite scheduling[J]. Journal of Electronics & Information Technology, 2025, 47(12): 5033–5047. doi: 10.11999/JEIT251038.
[7]	李庥甜, 王凌, 陈英武, 等. 基于自适应大邻域搜索的多场景多卫星任务规划方法[J]. 系统仿真学报, 2025, 37(7): 1836–1847. doi: 10.16182/j.issn1004731x.joss.25-0095. LI Xiutian, WANG Ling, CHEN Yingwu, et al. Multi-scenario multi-satellite mission planning method based on adaptive large neighborhood search[J]. Journal of System Simulation, 2025, 37(7): 1836–1847. doi: 10.16182/j.issn1004731x.joss.25-0095.
[8]	SHE Yucheng, YANG Zhi, and WANG Dandan. A study on adaptive multi-satellite mission allocation algorithm for efficient large-scale constellation planning[J]. Journal of Physics: Conference Series, 2025, 3083(1): 012002. doi: 10.1088/1742-6596/3083/1/012002.
[9]	HUANG Weiquan, WANG He, YI Dongbo, et al. A multiple agile satellite staring observation mission planning method for dense regions[J]. Remote Sensing, 2023, 15(22): 5317. doi: 10.3390/rs15225317.
[10]	秦嘉豪, 李宝卫, 白雪, 等. 面向异质卫星集群的事件触发分布式自主任务规划方法[J]. 中国空间科学技术(中英文), 2025, 45(4): 88–101. doi: 10.16708/j.cnki.1000-758X.2025.0061. QIN Jiahao, LI Baowei, BAI Xue, et al. Distributed autonomous scheduling based on event trigger for heterogeneous satellite swarm[J]. Chinese Space Science and Technology, 2025, 45(4): 88–101. doi: 10.16708/j.cnki.1000-758X.2025.0061.
[11]	HILTON S, THANGAVEL K, GARDI A, et al. Intelligent mission planning for autonomous distributed satellite systems[J]. Acta Astronautica, 2024, 225: 123–136. doi: 10.1016/j.actaastro.2024.08.050.
[12]	袁健波, 杜永浩, 陈盈果, 等. 面向点群与大区域目标的成像卫星任务规划模型与算法研究[J]. 系统工程与电子技术, 2025, 47(9): 2939–2950. doi: 10.12305/j.issn.1001-506X.2025.09.15. YUAN Jianbo, DU Yonghao, CHEN Yingguo, et al. Research on imaging satellite mission planning model and algorithm for point-cluster and large-region targets[J]. Systems Engineering and Electronics, 2025, 47(9): 2939–2950. doi: 10.12305/j.issn.1001-506X.2025.09.15.
[13]	LONG Jun, WU Shimin, HAN Xiaodong, et al. Autonomous task planning method for multi-satellite system based on a hybrid genetic algorithm[J]. Aerospace, 2023, 10(1): 70. doi: 10.3390/aerospace10010070.
[14]	CHEN Yaxin, SHEN Xin, ZHANG Guo, et al. Large-scale multi-objective imaging satellite task planning algorithm for vast area mapping[J]. Remote Sensing, 2023, 15(17): 4178. doi: 10.3390/rs15174178.
[15]	许可, 孙昌浩, 谢睿达, 等. 基于DQN的对地观测卫星调度算法[J]. 空间控制技术与应用(中英文), 2026, 52(1): 68–78. doi: 10.3969/j.issn.1674-1579.2026.01.007. XU Ke, SUN Changhao, XIE Ruida, et al. Earth observation satellite scheduling based on DQN[J]. Aerospace Control and Application, 2026, 52(1): 68–78. doi: 10.3969/j.issn.1674-1579.2026.01.007.
[16]	陈盈果, 王斐然, 胡云鹏, 等. 融合大语言模型与强化学习的敏捷卫星任务分配算法设计[J]. 电子与信息学报, 2025, 47(12): 4959–4972. doi: 10.11999/JEIT250991. CHEN Yingguo, WANG Feiran, HU Yunpeng, et al. Automating algorithm design for agile satellite task assignment with large language models and reinforcement learning[J]. Journal of Electronics & Information Technology, 2025, 47(12): 4959–4972. doi: 10.11999/JEIT250991.
[17]	WU Xiande, ZHANG Zehua, CHEN Zhengnan, et al. Intelligent task planning method for distributed satellite system based on reinforcement learning[J]. Aerospace Science and Technology, 2026, 168: 111325. doi: 10.1016/j.ast.2025.111325.
[18]	HE Xiaohe, XIANG Junyan, YAN Mubiao, et al. Integrated clustering and mission planning for agile Earth observation satellite constellations[J]. IEICE Transactions on Electronics, 2025, E108. C(11): 594–597. doi: 10.1587/transele.2024ECS6016.
[19]	LI Shuo, WANG Gang, and CHEN Jinyong. AEM-D3QN: A graph-based deep reinforcement learning framework for dynamic earth observation satellite mission planning[J]. Aerospace, 2025, 12(5): 420. doi: 10.3390/aerospace12050420.
[20]	杜永浩, 黎磊, 徐世龙, 等. 基于智能优化算法引擎的可演进星群智能任务规划[J]. 电子与信息学报, 2025, 47(6): 1645–1657. doi: 10.11999/JEIT240974. DU Yonghao, LI Lei, XU Shilong, et al. Evolutionary optimization for satellite constellation task scheduling based on intelligent optimization engine[J]. Journal of Electronics & Information Technology, 2025, 47(6): 1645–1657. doi: 10.11999/JEIT240974.
[21]	魏普远, 何磊. 基于深度强化学习的自适应大邻域搜索算法在成像卫星调度问题中的应用[J]. 电子与信息学报, 2025, 47(12): 5005–5015. doi: 10.11999/JEIT251009. WEI Puyuan and HE Lei. A deep reinforcement learning enhanced adaptive large neighborhood search for imaging satellite scheduling[J]. Journal of Electronics & Information Technology, 2025, 47(12): 5005–5015. doi: 10.11999/JEIT251009.
[22]	SAMVELYAN M, RASHID T, DE WITT C S, et al. The StarCraft multi-agent challenge[C]. The 18th International Conference on Autonomous Agents and MultiAgent Systems, Montreal, Canada, 2019.