Multi-objective Remote Sensing Product Production Task Scheduling Algorithm Based on Double Deep Q-Network

ZHOU Liming; YU Xi; FAN Minghu; ZUO Xianyu; QIAO Baojun

doi:10.11999/JEIT250089

Volume 47 Issue 8

Aug. 2025

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 > 47(8): 2819-2829

ZHOU Liming, YU Xi, FAN Minghu, ZUO Xianyu, QIAO Baojun. Multi-objective Remote Sensing Product Production Task Scheduling Algorithm Based on Double Deep Q-Network[J]. Journal of Electronics & Information Technology, 2025, 47(8): 2819-2829. doi: 10.11999/JEIT250089

Citation:

ZHOU Liming, YU Xi, FAN Minghu, ZUO Xianyu, QIAO Baojun. Multi-objective Remote Sensing Product Production Task Scheduling Algorithm Based on Double Deep Q-Network[J]. Journal of Electronics & Information Technology, 2025, 47(8): 2819-2829. doi: 10.11999/JEIT250089

Citation:

PDF( 3759 KB)

Multi-objective Remote Sensing Product Production Task Scheduling Algorithm Based on Double Deep Q-Network

doi: 10.11999/JEIT250089 cstr: 32379.14.JEIT250089

1.
Henan Key Laboratory of Big Data Analysis and Processing, Henan University, Kaifeng 475004, China
2.
School of Computer and Information Engineering, Henan University, Kaifeng 475004, China

Funds: Henan Province University Science and Technology Innovation Team Support Plan (24IRTSTHN021), The Key Research and Promotion Projects of Henan Province (242102210081, 252102211053), The Postgraduate Education Reform and Quality Improvement Project of Henan Province (YJS2023JD28)

Received Date: 2025-02-17
Rev Recd Date: 2025-06-15

Available Online: 2025-06-24

Publish Date: 2025-08-27

Abstract

Abstract

Objective Remote sensing product generation is a multi-task scheduling problem influenced by dynamic factors, including resource contention and real-time environmental changes. Achieving adaptive, multi-objective, and efficient scheduling remains a central challenge. To address this, a Multi-Objective Remote Sensing scheduling algorithm (MORS) based on a Double Deep Q-Network (DDQN) is proposed. A subset of candidate algorithms is first identified using a value-driven, parallel-executable screening strategy. A deep neural network is then designed to perceive the characteristics of both remote sensing algorithms and computational nodes. A reward function is constructed by integrating algorithm execution time and node resource status. The DDQN is employed to train the model to select optimal execution nodes for each algorithm in the processing subset. This approach reduces production time and enables load balancing across computational nodes. Methods The MORS scheduling process comprises two stages: remote sensing product processing and screening, followed by scheduling model training and execution. A time-triggered strategy is adopted, whereby all newly arrived remote sensing products within a predefined time window are collected and placed in a task queue. For efficient scheduling, each product is parsed into a set of executable remote sensing algorithms. Based on the model illustrated in Figure 2, the processing unit extracts all constituent algorithms to form an algorithm set. An optimal subset is then selected using a value-driven parallel-executable screening strategy. The scheduling process is modeled as a Markov decision process, and the DDQN is applied to assign each algorithm in the selected subset to the optimal virtual node. Results and Discussions Simulation experiments use varying numbers of tasks and nodes to evaluate the performance of MORS. Comparative analyses are conducted against several baseline scheduling algorithms, including First-Come, First-Served (FCFS), Round Robin (RR), Genetic Algorithm (GA), Deep Q-Network (DQN), and Dueling Deep Q-Network (Dueling DQN). The results demonstrate that MORS outperforms all other algorithms in terms of scheduling efficiency and adaptability in remote sensing task scheduling. The learning rate, a critical hyperparameter in DDQN, influences the step size for parameter updates during training. When the learning rate is set to 0.00001, the model fails to converge even after 5,000 iterations due to extremely slow optimization. A learning rate of 0.0001 achieves a balance between convergence speed and training stability, avoiding oscillations associated with overly large learning rates (Figure 3 and Figure 4). The corresponding DDQN loss values show a steady decline, reflecting effective optimization and gradual convergence. In contrast, the unpruned DDQN initially declines sharply but plateaus prematurely, failing to reach optimal convergence. DDQN without soft updates shows large fluctuations in loss and remains unstable during later training stages, indicating that the absence of soft updates impairs convergence (Figure 5). Regarding decision quality, the reward values of DDQN gradually approach 25 in the later training stages, reflecting stable convergence and strong decision-making performance. Conversely, DDQN models without pruning or soft updates display unstable reward trajectories, particularly the latter, which exhibits pronounced reward fluctuations and slower convergence (Figure 6). A comparison of DQN, Dueling DQN, and DDQN reveals that all three show decreasing training loss, suggesting continuous optimization (Figure 7). However, the reward curve of Dueling DQN shows higher volatility and reduced stability (Figure 8). To further assess scalability, four sets of simulation experiments use 30, 60, 90, and 120 remote sensing tasks, with the number of virtual machine nodes fixed at 15. Each experimental configuration is evaluated using 100 Monte Carlo iterations to ensure statistical robustness. DDQN consistently shows superior performance under high-concurrency conditions, effectively managing increased scheduling pressure (Table 7). In addition, DDQN exhibits lower standard deviations in node load across all task volumes, reflecting more balanced resource allocation and reduced fluctuation in system utilization (Table 8 and Table 9). Conclusions The proposed MORS algorithm addresses the variability and complexity inherent in remote sensing task scheduling. Experimental results demonstrate that MORS not only improves scheduling efficiency but also significantly reduces production time and achieves balanced allocation of node resources.
- Remote sensing task scheduling,
- Multi-objective optimization,
- Double Deep Q-Network (DDQN)

FullText(HTML)

References(31)

References

[1]	AHMAD R. Smart remote sensing network for disaster management: An overview[J]. Telecommunication Systems, 2024, 87(1): 213–237. doi: 10.1007/s11235-024-01148-z.
[2]	孙伟伟, 苏奋振, 侯西勇, 等. 遥感助力海岸带可持续发展—首届全国海岸带遥感大会总结[J]. 遥感学报, 2024, 28(4): 1123–1128. doi: 10.11834/jrs.20243538. SUN Weiwei, SUN Fenzhen, HOU Xiyong, et al. Remote sensing for sustainable development of coastal Zones-Summary of the first national conference on remote sensing of coastal zones[J]. National Remote Sensing Bulletin, 2024, 28(4): 1123–1128. doi: 10.11834/jrs.20243538.
[3]	XING Xiaoyue, YU Bailang, KANG Chaogui, et al. The synergy between remote sensing and social sensing in urban studies: Review and perspectives[J]. IEEE Geoscience and Remote Sensing Magazine, 2024, 12(1): 108–137. doi: 10.1109/MGRS.2023.3343968.
[4]	WANG Fumin, LI Jiale, PENG Dailiang, et al. Estimating soybean yields using causal inference and deep learning approaches with satellite remote sensing data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024, 17: 14161–14178. doi: 10.1109/JSTARS.2024.3435699.
[5]	GROSOF I, YANG Kunhe, SCULLY Z, et al. Nudge: Stochastically improving upon FCFS[C]. 2021 ACM SIGMETRICS/International Conference on Measurement and Modeling of Computer Systems, 2021: 11–12. doi: 10.1145/3410220.3460102.
[6]	PACHIPALA Y, SUREDDY K S, SRIYA KAITEPALLI A B S, et al. Optimizing task scheduling in cloud computing: An enhanced shortest job first algorithm[J]. Procedia Computer Science, 2024, 233: 604–613. doi: 10.1016/j.procs.2024.03.250.
[7]	BUSENHART C, HUNGERBÜHLER N, and XU W. A Variant of the round-robin scheduling problem[J]. Ars Combinatoria, 2024, 158: 81–92. doi: 10.61091/ars158-09.
[8]	GAWIEJNOWICZ S, LIN B M T, and MOSHEIOV G. Dynamic scheduling problems in theory and practice[J]. Journal of Scheduling, 2024, 27(3): 225–226. doi: 10.1007/s10951-023-00798-2.
[9]	ABO-ALSABEH R and SALHI A. The genetic algorithm: A study survey[J] Iraqi Journal of Science, 2022, 63(3): 1215–1231. doi: 10.24996/ijs.2022.63.3.27.
[10]	陈亮, 李景山. 遥感卫星数据预处理系统复杂任务调度模型研究与实现[J]. 遥感信息, 2021, 36(5): 98–106. doi: 10.3969/j.issn.1000-3177.2021.05.014. CHEN Liang and LI Jingshan. Research and implementation of complex task scheduling model for remote sensing satellite data preprocessing system[J]. Remote Sensing Information, 2021, 36(5): 98–106. doi: 10.3969/j.issn.1000-3177.2021.05.014.
[11]	赵斐, 陈昊, 白建东, 等. 基于改进蚁群算法的遥感信息处理负载均衡任务调度算法研究[J]. 计算机测量与控制, 2021, 29(11): 183–188. doi: 10.16526/j.cnki.11-4762/tp.2021.11.033. ZHAO Fei, CHEN Hao, BAI Jiandong, et al. One remote sensing information processing task allocation algorithm for collaborative planning based on knowledge model and genetic algorithm[J]. Computer Measurement & Control, 2021, 29(11): 183–188. doi: 10.16526/j.cnki.11-4762/tp.2021.11.033.
[12]	ZHANG Jianxiong, GUO Bing, DING Xuefeng, et al. An adaptive multi-objective multi-task scheduling method by hierarchical deep reinforcement learning[J]. Applied Soft Computing, 2024, 154: 111342. doi: 10.1016/j.asoc.2024.111342.
[13]	LI Guohao, LI Xuefei, LI Jing, et al. PTMB: An online satellite task scheduling framework based on pre-trained Markov decision process for multi-task scenario[J]. Knowledge-Based Systems, 2024, 284: 111339. doi: 10.1016/j.knosys.2023.111339.
[14]	HAN Xuefeng, HE Hongwen, WU Jingda, et al. Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle[J]. Applied Energy, 2019, 254: 113708. doi: 10.1016/j.apenergy.2019.113708.
[15]	李奇儒, 耿霞. 基于改进DQN算法的机器人路径规划[J]. 计算机工程, 2023, 49(12): 111–120. doi: 10.19678/j.issn.1000-3428.0066348. LI Qiru and GENG Xia. Robot path planning based on improved DQN algorithm[J]. Computer Engineering, 2023, 49(12): 111–120. doi: 10.19678/j.issn.1000-3428.0066348.
[16]	WANG Xinwei, WU Guohua, XING Lining, et al. Agile earth observation satellite scheduling over 20 years: Formulations, methods, and future directions[J]. IEEE Systems Journal, 2021, 15(3): 3881–3892. doi: 10.1109/JSYST.2020.2997050.
[17]	胡庆雷, 邵小东, 杨昊旸, 等. 航天器多约束姿态规划与控制: 进展与展望[J]. 航空学报, 2022, 43(10): 527351. doi: 10.7527/S1000-6893.2022.27351. HU Qinglei, SHAO Xiaodong, YANG Haoyang, et al. Spacecraft attitude planning and control under multiple constraints: Review and prospects[J]. Acta Aeronautica et Astronautica Sinica, 2022, 43(10): 527351. doi: 10.7527/S1000-6893.2022.27351.
[18]	何奇恩, 李峰, 钟兴. 多目标算法在卫星区域覆盖调度及数传规划上的应用综述[J]. 遥感技术与应用, 2023, 38(4): 783–793. doi: 10.11873/j.issn.1004-0323.2023.4.0783. HE Qien, LI Feng, and ZHONG Xing. A review of the application of multi-objective algorithms in satellite regional coverage scheduling and data transmission planning[J]. Remote Sensing Technology and Application, 2023, 38(4): 783–793. doi: 10.11873/j.issn.1004-0323.2023.4.0783.
[19]	DROZDOWSKI M. Scheduling multiprocessor tasks—An overview[J]. European Journal of Operational Research, 1996, 94(2): 215–230. doi: 10.1016/0377-2217(96)00123-3.
[20]	杨戈, 赵鑫, 黄静. 云环境下调度算法综述[J]. 电子技术应用, 2019, 45(9): 13–17,27. doi: 10.16157/j.issn.0258-7998.190547. YANG Ge, ZHAO Xin, and HUANG Jing. Overview of task scheduling algorithms in cloud computing[J]. Application of Electronic Technique, 2019, 45(9): 13–17,27. doi: 10.16157/j.issn.0258-7998.190547.
[21]	ELZEKI O M, RASHAD M Z, and ELSOUD M A. Overview of scheduling tasks in distributed computing systems[J]. International Journal of Soft Computing and Engineering (IJSCE), 2012, 2(3): 470–475.
[22]	KUCHUK H, MOZHAIEV O, KUCHUK N, et al. Devising a method for the virtual clustering of the internet of things edge environment[J]. Eastern-European Journal of Enterprise Technologies, 2024, 1(9): 60–71. doi: 10.15587/1729-4061.2024.298431.
[23]	WANG Jin. Virtual cluster model based on big data application[M]. LI Xiaolong. Advances in Intelligent Automation and Soft Computing. Cham: Springer, 2022: 61–66. doi: 10.1007/978-3-030-81007-8_8.
[24]	AGHAEI M, ASGHARI P, ADABI S, et al. Using recommender clustering to improve quality of services with sustainable virtual machines in cloud computing[J]. Cluster Computing, 2023, 26(2): 1479–1493. doi: 10.1007/s10586-022-03760-7.
[25]	GHASEMI A and KESHAVARZI A. Energy-efficient virtual machine placement in heterogeneous cloud data centers: A clustering-enhanced multi-objective, multi-reward reinforcement learning approach[J]. Cluster Computing, 2024, 27(10): 14149–14166. doi: 10.1007/s10586-024-04657-3.
[26]	HAN Xiaoyun, MU Chaoxu, ZHU Jiebei, et al. A safe virtual machine scheduling strategy for energy conservation and privacy protection of server clusters in cloud data centers[J]. IEEE Transactions on Sustainable Computing, 2024, 9(1): 46–60. doi: 10.1109/TSUSC.2023.3303637.
[27]	ZHU Xi, ZHANG Lei, and TANG Shaoqiang. Adaptive selection of reference stiffness in virtual clustering analysis[J]. Computer Methods in Applied Mechanics and Engineering, 2021, 376: 113621. doi: 10.1016/j.cma.2020.113621.
[28]	CHRAIBI A, BEN ALLA S, TOUHAFI A, et al. A novel dynamic multi-objective task scheduling optimization based on Dueling DQN and PER[J]. The Journal of Supercomputing, 2023, 79(18): 21368–21423. doi: 10.1007/s11227-023-05489-5.
[29]	MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529–533. doi: 10.1038/nature14236.
[30]	KOBAYASHI T and ILBOUDO W E L. t-soft update of target network for deep reinforcement learning[J]. Neural Networks, 2021, 136: 63–71. doi: 10.1016/j.neunet.2020.12.023.
[31]	HUANG Heqing, CHIU H C, SHI Qingkai et al. Balance seed scheduling via Monte Carlo planning[J]. IEEE Transactions on Dependable and Secure Computing, 2024, 21(3): 1469–1483. doi: 10.1109/TDSC.2023.3285293.