天地一体化边缘计算网络服务迁移算法研究

冯伊凡; 吴畏虹; 孙罡; 王颖; 罗龙; 虞红芳

doi:10.11999/JEIT250835

天地一体化边缘计算网络服务迁移算法研究

doi: 10.11999/JEIT250835 cstr: 32379.14.JEIT250835

冯伊凡¹,
吴畏虹^1, ,,
孙罡¹,
王颖²,
罗龙¹,
虞红芳¹

1.
电子科技大学成都 611731
2.
紫金山实验室南京 211111

详细信息

作者简介:
冯伊凡：女，硕士生，研究方向为天地一体化网络

吴畏虹：男，副研究员，研究方向为新一代网络

孙罡：男，教授，研究方向为网络虚拟化、区块链技术、人工智能和网络系统安全

王颖：女，博士后，研究方向为卫星网络

罗龙：女，副教授，研究方向为算力网络，智算网络资源调度

虞红芳：女，教授，研究方向为智慧网络及应用研究

通讯作者:
吴畏虹　wuweihong@uestc.edu.cn

中图分类号: TN92
计量
- 文章访问数: 57
- HTML全文浏览量: 12
- PDF下载量: 16
- 被引次数: 0
出版历程
- 收稿日期: 2025-09-01
- 修回日期: 2025-12-19
- 录用日期: 2025-12-19
- 网络出版日期: 2025-12-23

Service Migration Algorithm for Satellite-terrestrial Edge Computing Networks

1.
University of Electronic Science and Technology of China, Chengdu 611731, China
2.
Purple Mountain Laboratories, Nanjing 211111, China

摘要

摘要: 针对天地一体化边缘计算网络(STECN)的高动态性和复杂性，如何协同优化用户服务延迟与系统迁移成本成为服务迁移算法设计的关键问题。因此，该文提出一种多智能体服务迁移优化(MASMO)算法。首先，考虑到低轨卫星的有限覆盖时间、网络拓扑的动态变化和卫星节点资源等多重因素，对用户服务延迟和系统迁移成本进行建模。其次，将服务迁移优化问题进一步建模为多智能体马尔可夫决策过程(MAMDP)。随后，采用基于轨迹感知的状态信息增强方法，通过融合卫星轨道的可预测信息，引导智能体学习具备前瞻性与稳定性的迁移行为。最后，基于循环多智能体近端策略优化(rMAPPO)算法对服务迁移优化问题进行求解，以最大程度地降低用户服务延迟和系统长期迁移成本。仿真结果表明，所提算法具有良好的收敛性，能够有效协调服务延迟与迁移成本之间的矛盾，对用户服务延迟降低2.90%$ \sim $14.63%的同时，有效降低了系统服务迁移成本11.39%$ \sim $30.57%。
- 天地一体化网络 /
- 服务迁移 /
- 多智能体深度强化学习
Abstract: Objective In highly dynamic Satellite-Terrestrial Edge Computing Networks (STECN), achieving coordinated optimization between user service latency and system migration cost is a central challenge in service migration algorithm design. Existing approaches often fail to maintain stable performance in such environments. To address this, a Multi-Agent Service Migration Optimization (MASMO) algorithm based on multi-agent deep reinforcement learning is proposed to provide an intelligent and forward-looking solution for dynamic service management in STECN. Methods The service migration optimization problem is formulated as a Multi-Agent Markov Decision Process (MAMDP), which offers a framework for sequential decision-making under uncertainty. The environment represents the spatiotemporal characteristics of a Low Earth Orbit (LEO) satellite network, where satellite movement and satellite-user visibility define time-varying service availability. Service latency is expressed as the sum of transmission delay and computation delay. Migration cost is modeled as a function of migration distance between satellite nodes to discourage frequent or long-range migrations. A Trajectory-Aware State Enhancement (TASE) method is proposed to incorporate predictable orbital information of LEO satellites into the agent state representation, improving proactive and stable migration actions. Optimization is performed using the recurrent Multi-Agent Proximal Policy Optimization (rMAPPO) algorithm, which is suitable for cooperative multi-agent tasks. The reward function balances the objectives by penalizing high migration cost and rewarding low service latency. Results and Discussions Simulations are conducted in dynamic STECN scenarios to compare MASMO with MAPPO, MADDPG, Greedy, and Random strategies. The results consistently confirm the effectiveness of MASMO. As the number of users increases, MASMO shows slower performance degradation. With 16 users, it reduces average service latency by 2.90%, 6.78%, 11.01%, and 14.63% compared with MAPPO, MADDPG, Greedy, and Random. It also maintains high cost efficiency, lowering migration cost by up to 14.69% at 12 users (Fig. 3). When satellite resources increase, MASMO consistently leverages the added availability to reduce both latency and migration cost, whereas myopic strategies such as Greedy do not exhibit similar improvements. With 10 satellites, MASMO achieves the lowest service latency and outperforms the next-best method by 7.53% (Fig. 4). These findings show that MASMO achieves an effective balance between transmission latency and migration latency through its forward-looking decision policy. Conclusions This study addresses the service migration challenge in STECN through the MASMO algorithm, which integrates the TASE method with rMAPPO. The method improves service latency and reduces migration cost at the same time, demonstrating strong performance advantages. The trajectory-enhanced state representation improves foresight and stability of migration behavior in predictable dynamic environments. This study assumes ideal real-time state perception, and future work should evaluate communication delays and partial observability, as well as investigate scalability in larger satellite constellations with heterogeneous user demands.
- Satellite-Terrestrial Edge Computing Network (STECN) /
- Service migration /
- Multi-Agent Reinforcement Learning (MARL)

HTML全文

图 1 天地一体化边缘计算网络服务迁移场景

下载: 全尺寸图片幻灯片

图 2 以卫星$ {s}_{i} $为中心的局部TEG及动态MDP

下载: 全尺寸图片幻灯片

图 3 MASMO算法的性能分析

下载: 全尺寸图片幻灯片

图 4 用户数量对算法性能的影响

下载: 全尺寸图片幻灯片

图 5 卫星数量对算法性能的影响

下载: 全尺寸图片幻灯片

1 MASMO算法

输入：环境状态$ {O}_{t} $
输出：最优服务迁移策略$ {\pi }_{\theta } $
1) 初始化actor网络$ {\pi }_{\theta } $与critic网络$ {V}_{\phi } $，初始化经验回放缓冲区　$ D $。
2) for 训练迭代次数$ k=1,2,\cdots ,K $do
3) 　清空经验回放缓冲区$ D $。
4) 　重置并行环境$ e=1,2,\cdots ,{N}_{{\mathrm{env}}} $
5) 　for 并行环境$ e=1,2,\cdots, {N}_{{\mathrm{env}}} $ do
6) 　　for $ t=1,2,\cdots ,T $ do
7) 　　基于局部观测与预测增强特征构造状态$ \boldsymbol{o}_{\mathrm{joint},t} $。
8) 　　通过执行当前策略$ {\pi }_{\theta } $与环境交互，采集一条经验轨迹　$ {\tau }_{e} $。
9) 　　将轨迹$ {\tau }_{e} $存入经验回放缓冲区$ D $。
10) end for
11) 保存当前策略参数$ {\theta }_{{\mathrm{old}}} $←$ \theta $。
12) 利用critic网络$ {V}_{\phi } $和采集到的数据计算优势估计和$ \hat{A} $回报目　标$\hat {{R}} $。
13) for 更新轮次 u=1,2,···,U do
14) 　从$ D $中随机抽取n个经验作为一个mini-batch b；
15) 　对于 mini-batch $ b $中的每个数据块 $ c $ do
16) 　　使用数据块首帧的隐藏状态更新$ \pi $和V的RNN状态。
17) 　通过最小化损失函数$ L\left(\phi \right) $更新critic参数$ \phi $。
18) 　通过最大化目标函数$ J(\theta ) $更新actor参数$ \theta $。
19) end for
20) end for

下载: 导出CSV

表 1 参数设置

参数	值
卫星轨道高度$ h $ (km)	212
卫星轨道面倾角(°)	51.67
卫星节点所配备的计算资源 (Gcycles/s)	[1 000, 2 000]
卫星节点所配备的存储资源(GB)	[2, 3]
地面用户坐标经度范围(°)	[30, 50]
地面用户坐标纬度范围(°)	[100, 135]
最小仰角(°)	30
地面用户请求任务大小(kB)	[200, 500]
服务实体大小$ {I}_{e} $ (MB)	[300, 500]
任务请求所需要的计算资源 (Gcycle/s)	[100, 200]
任务请求所需要的计算强度 (cycle/bit)	10⁶
地面用户设备的发射功率$ {P}_{u} $(W)	5
地面用户到卫星之间的信道带宽 (MHz)	8
地面用户设备天线发射增益 (dBi)	5
卫星天线发射增益 (dBi)	20
卫星天线接收增益 (dBi)	40
实验场景周期 $ T $(s)	600
单个时隙 $ t $ (s)	20
K	4
$ {\omega }_{1},{\omega }_{2} $	0.6, 0.4
训练轮次	1800
经验回放缓冲区大小	10⁵
学习率l_r	5e–5
折扣因子$ \gamma $	0.95
$ \lambda $	0.95
mini-batch大小	80

下载: 导出CSV

参考文献(24)

[1]	高媛, 方海, 赵扬, 等. 基于自然梯度Actor-Critic强化学习的卫星边缘网络服务功能链部署方法[J]. 电子与信息学报, 2023, 45(2): 455–463. doi: 10.11999/JEIT211384. GAO Yuan, FANG Hai, ZHAO Yang, et al. A satellite edge network service function chain deployment method based on natural gradient Actor-Critic reinforcement learning[J]. Journal of Electronics & Information Technology, 2023, 45(2): 455–463. doi: 10.11999/JEIT211384.
[2]	JIA Min, ZHANG Liang, WU Jian, et al. Joint computing and communication resource allocation for edge computing towards Huge LEO networks[J]. China Communications, 2022, 19(8): 73–84. doi: 10.23919/JCC.2022.08.006.
[3]	WANG Shangguang and LI Qing. Satellite computing: Vision and challenges[J]. IEEE Internet of Things Journal, 2023, 10(24): 22514–22529. doi: 10.1109/JIOT.2023.3303346.
[4]	王鹏, 张佳鑫, 张兴, 等. 低轨卫星智能多接入边缘计算网络: 需求、架构、机遇与挑战[J]. 移动通信, 2021, 45(5): 35–46. doi: 10.3969/j.issn.1006-1010.2021.05.007. WANG Peng, ZHANG Jiaxin, ZHANG Xing, et al. Low earth orbit satellite intelligent multi-access edge computing networks: Requirements, architecture, opportunities and challenges[J]. Mobile Communication, 2021, 45(5): 35–46. doi: 10.3969/J.ISSN.1006-1010.2021.05.007. doi: 10.3969/j.issn.1006-1010.2021.05.007.
[5]	ZHOU Jian, YANG Qi, ZHAO Lu, et al. Mobility-aware computation offloading in satellite edge computing networks[J]. IEEE Transactions on Mobile Computing, 2024, 23(10): 9135–9149. doi: 10.1109/TMC.2024.3359759.
[6]	曹怡璐, 贾子晔, 尤嘉豪, 等. 基于SDN和NFV的空天地一体化网络任务部署与恢复综述[J]. 电信科学, 2025, 41(5): 1–16. doi: 10.11959/j.issn.1000-0801.2025138. CAO Yilu, JIA Ziye, YOU Jiahao, et al. A survey of task deployment and recovery in space-air-ground integrated networks based on SDN and NFV[J]. Telecommunications Science, 2025, 41(5): 1–16. doi: 10.11959/j.issn.1000-0801.2025138.
[7]	XIE Renchao, TANG Qinqin, WANG Qiuning, et al. Satellite-terrestrial integrated edge computing networks: Architecture, challenges, and open issues[J]. IEEE Network, 2020, 34(3): 224–231. doi: 10.1109/MNET.011.1900369.
[8]	DENG Peng, GONG Xiangyang, and QUE Xirong. A bandwidth-aware service migration method in LEO satellite edge computing network[J]. Computer Communications, 2023, 200: 104–112. doi: 10.1016/j.comcom.2023.01.007.
[9]	HE Lijun, JIA Ziye, GUO Kun, et al. Online joint data offloading and power control for space-air-ground integrated networks[J]. IEEE Transactions on Wireless Communications, 2024, 23(12): 18126–18141. doi: 10.1109/TWC.2024.3462349.
[10]	JIA Ziye, CAO Yilu, HE Lijun, et al. NFV-enabled service recovery in space–air–ground integrated networks: A matching game-based approach[J]. IEEE Transactions on Network Science and Engineering, 2025, 12(3): 1732–1744. doi: 10.1109/TNSE.2025.3538614.
[11]	WANG Houpeng, GAO Yu’e, GUO Zhonglin, et al. Dynamic service migration mechanism in satellite edge computing with location privacy protection[C]. 2024 IEEE 24th International Conference on Communication Technology (ICCT), Chengdu, China, 2024: 1073–1080. doi: 10.1109/ICCT62411.2024.10946657.
[12]	LI Ziqi, ZHANG Heli, LIU Chunyu, et al. Online service deployment on mega-LEO satellite constellations for end-to-end delay optimization[J]. IEEE Transactions on Network Science and Engineering, 2024, 11(1): 1214–1226. doi: 10.1109/TNSE.2023.3321644.
[13]	KSENTINI A, TALEB T, and CHEN Min. A Markov Decision Process-based service migration procedure for follow me cloud[C]. 2014 IEEE International Conference on Communications (ICC), Sydney, Australia, 2014: 1350–1354. doi: 10.1109/ICC.2014.6883509.
[14]	WU Haonan, YANG Xiumei, and BU Zhiyong. Task offloading with service migration for satellite edge computing: A deep reinforcement learning approach[J]. IEEE Access, 2024, 12: 25844–25856. doi: 10.1109/ACCESS.2024.3367128.
[15]	LI Zhen, JIANG Chunxiao, and LU Jianhua. Distributed service migration in satellite mobile edge computing[C]. 2021 IEEE Global Communications Conference (GLOBECOM), Madrid, Spain, 2021: 1–6. doi: 10.1109/GLOBECOM46510.2021.9685350.
[16]	SUN Jiayu, WANG Huiqiang, NIE Lili, et al. A joint strategy for service deployment and task offloading in satellite–terrestrial IoT[J]. Computer Networks, 2023, 225: 109656. doi: 10.1016/j.comnet.2023.109656.
[17]	JIA Ziye, CAO Yilu, HE Lijun, et al. Service function chain dynamic scheduling in space-air-ground integrated networks[J]. IEEE Transactions on Vehicular Technology, 2025, 74(7): 11235–11248. doi: 10.1109/TVT.2025.3543259.
[18]	WANG Xu, JU Xiaojie, XIE Renchao, et al. Service continuity guarantee for coordinated optimization of offloading and migration in LEO satellite computing power networks[C]. 2024 10th International Conference on Computer and Communications (ICCC), Chengdu, China, 2024: 1857–1862. doi: 10.1109/ICCC62609.2024.10942057.
[19]	LI Qing, WANG Shangguang, MA Xiao, et al. Service coverage for satellite edge computing[J]. IEEE Internet of Things Journal, 2022, 9(1): 695–705. doi: 10.1109/JIOT.2021.3085129.
[20]	YU Kangjia, CUI Qimei, LYU Xinchen, et al. Efficient collaborative computing for multilayer LEO satellites with spatiotemporal dynamics: A long-term continuous timescale optimization[J]. IEEE Internet of Things Journal, 2025, 12(6): 7459–7471. doi: 10.1109/JIOT.2024.3498322.
[21]	BHATTACHERJEE D and SINGLA A. Network topology design at 27, 000 km/hour[C]. The 15th International Conference on Emerging Networking Experiments and Technologies, Orlando, Florida, 2019: 341–354. doi: 10.1145/3359989.3365407.
[22]	WANG Shiqiang, URGAONKAR R, HE Ting, et al. Dynamic service placement for mobile micro-clouds with predicted future costs[J]. IEEE Transactions on Parallel and Distributed Systems, 2017, 28(4): 1002–1016. doi: 10.1109/TPDS.2016.2604814.
[23]	WANG Shiqiang, URGAONKAR R, ZAFER M, et al. Dynamic service migration in mobile edge computing based on Markov decision process[J]. IEEE/ACM Transactions on Networking, 2019, 27(3): 1272–1288. doi: 10.1109/TNET.2019.2916577.
[24]	YUAN Quan, LI Jinglin, ZHOU Haibo, et al. A joint service migration and mobility optimization approach for vehicular edge computing[J]. IEEE Transactions on Vehicular Technology, 2020, 69(8): 9041–9052. doi: 10.1109/TVT.2020.2999617.