Optimized Deployment Method of Edge Computing Network Service Function Chain Delay Combined with Deep Reinforcement Learning

SUN Chunxia; YANG Li; WANG Xiaopeng; LONG Liang

doi:10.11999/JEIT230632

Volume 46 Issue 4

Apr. 2024

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2024 > 46(4): 1363-1372

SUN Chunxia, YANG Li, WANG Xiaopeng, LONG Liang. Optimized Deployment Method of Edge Computing Network Service Function Chain Delay Combined with Deep Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2024, 46(4): 1363-1372. doi: 10.11999/JEIT230632

Citation:

SUN Chunxia, YANG Li, WANG Xiaopeng, LONG Liang. Optimized Deployment Method of Edge Computing Network Service Function Chain Delay Combined with Deep Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2024, 46(4): 1363-1372. doi: 10.11999/JEIT230632

Citation:

SUN Chunxia, YANG Li, WANG Xiaopeng, LONG Liang. Optimized Deployment Method of Edge Computing Network Service Function Chain Delay Combined with Deep Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2024, 46(4): 1363-1372. doi: 10.11999/JEIT230632

PDF( 3733 KB)

Optimized Deployment Method of Edge Computing Network Service Function Chain Delay Combined with Deep Reinforcement Learning

doi: 10.11999/JEIT230632

School of Electronics and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China

Funds: Gansu Province Higher Education Industry Support Plan Project (2023CYZC-40), Gansu Province Excellent Graduate 'Innovation Star' Program (2023CXZX-546)

Received Date: 2023-06-26
Rev Recd Date: 2023-11-06

Available Online: 2023-11-13

Publish Date: 2024-04-24

Abstract

Abstract

A delay-optimized Service Function Chain (SFC) deployment approach is proposed by combining deep reinforcement learning with the delay-based Dijkstra pathfinding algorithm for the problem of resource-constrained edge networks and low end-to-end delay tolerance for service flows. Firstly, an attention mechanism-based Sequence to Sequence (Seq2Seq) agent network and a delay-based Dijkstra pathfinding algorithm are designed for generating Virtual Network Function(VNF) deployments and link mapping for SFC, while the constraint problem of the delay optimization model is considered and incorporated into the reinforcement learning objective function using Lagrangian relaxation techniques; Secondly, to assist the network agent in converging quickly, a baseline evaluator network is used to assess the expected reward value of the deployment strategy; Finally, in the testing phase, the deployment strategy of the agent is improved by reducing the probability of convergence of the network to a local optimum through greedy search and sampling techniques. Comparison experiments show that the method reduces the latency by about 10% and 86.3% than the First-Fit algorithm and TabuSearch algorithm, respectively, and is about 74.2% and 84.4% more stable than these two algorithms in the case of limited network resources. This method provides a more stable end-to-end service with lower latency, enabling a better experience for latency-sensitive services.
- Service Function Chain(SFC) deployment,
- Deep reinforcement learning,
- Edge networks,
- End-to-end latency

FullText(HTML)

References(18)

References

[1]	陈卓, 冯钢, 刘怡静, 等. MEC中基于改进遗传模拟退火算法的虚拟网络功能部署策略[J]. 通信学报, 2020, 41(4): 70–80. doi: 10.11959/j.issn.1000-436x.2020074. CHEN Zhuo, FENG Gang, LIU Yijing, et al. Virtual network function deployment strategy based on improved genetic simulated annealing algorithm in MEC[J]. Journal on Communications, 2020, 41(4): 70–80. doi: 10.11959/j.issn.1000-436x.2020074.
[2]	ALLAHVIRDI A, YOUSEFI S, and SARDROUD A A. Placement of dynamic service function chains in partially VNF-enabled networks[J]. Telecommunication Systems, 2022, 81(2): 225–240. doi: 10.1007/s11235-022-00939-6.
[3]	YANG Song, LI Fan, YAHYAPOUR R, et al. Delay-sensitive and availability-aware virtual network function scheduling for NFV[J]. IEEE Transactions on Services Computing, 2022, 15(1): 188–201. doi: 10.1109/TSC.2019.2927339.
[4]	YAGHOUBPOUR F, BAKHSHI B, and SEIFI F. End-to-end delay guaranteed Service Function Chain deployment: A multi-level mapping approach[J]. Computer Communications, 2022, 194: 433–445. doi: 10.1016/j.comcom.2022.08.005.
[5]	唐伦, 李师锐, 杜雨聪, 等. 基于多智能体柔性演员-评论家学习的服务功能链部署算法[J]. 电子与信息学报, 2022, 45(8): 2893–2901. doi: 10.11999/JEIT220803. TANG Lun, LI Shirui, DU Yucong, et al. Deployment algorithm of service function chain based on multi-agent soft actor-critic learning[J]. Journal of Electronics & Information Technology, 2022, 45(8): 2893–2901. doi: 10.11999/JEIT220803.
[6]	LIU Hongtai, DING Shengduo, WANG Shunyi, et al. Multi-objective optimization service function chain placement algorithm based on reinforcement learning[J]. Journal of Network and Systems Management, 2022, 30(4): 58. doi: 10.1007/s10922-022-09673-5.
[7]	徐泽汐, 庄雷, 张坤丽, 等. 基于知识图谱的服务功能链在线部署算法[J]. 通信学报, 2022, 43(8): 41–51. doi: 10.11959/j.issn.1000-436x.2022154. XU Zexi, ZHUANG Lei, ZHANG Kunli, et al. Online placement algorithm of service function chain based on knowledge graph[J]. Journal on Communications, 2022, 43(8): 41–51. doi: 10.11959/j.issn.1000-436x.2022154.
[8]	HU Haiyan, KANG Qiaoyan, ZHAO Shuo, et al. Service function chain deployment method based on traffic prediction and adaptive virtual network function scaling[J]. Electronics, 2022, 11(16): 2625. doi: 10.3390/electronics11162625.
[9]	YALA L, FRANGOUDIS P A, and KSENTINI A. Latency and availability driven VNF placement in a MEC-NFV environment[C]. 2018 IEEE Global Communications Conference (GLOBECOM), Abu Dhabi, United Arab Emirates, 2018: 1–7. doi: 10.1109/GLOCOM.2018.8647858.
[10]	石尚. 蜂窝移动网络下服务功能链的部署与动态配置研究[D]. [硕士论文], 北京邮电大学, 2021. doi: 10.26969/d.cnki.gbydu.2021.002972. SHI Shang. Research on deployment and dynamic configuration of service function chain in mobile cellular network[D]. [Master dissertation], Beijing University of Posts and Telecommunications, 2021. doi: 10.26969/d.cnki.gbydu.2021.002972.
[11]	NAM Y, SONG S, and CHUNG J M. Clustered NFV Service chaining optimization in mobile edge clouds[J]. IEEE Communications Letters, 2017, 21(2): 350–353. doi: 10.1109/LCOMM.2016.2618788.
[12]	ADDIS B, BELABED D, BOUET M, et al. Virtual network functions placement and routing optimization[C]. 2015 IEEE 4th International Conference on Cloud Networking (CloudNet), Niagara Falls, Canada, 2015: 171–177. doi: 10.1109/CloudNet.2015.7335301.
[13]	MIJUMBI R, SERRAT J, GORRICHO J L, et al. Design and evaluation of algorithms for mapping and scheduling of virtual network functions[C]. The 2015 1st IEEE Conference on Network Softwarization (NetSoft), London, UK, 2015: 1–9. doi: 10.1109/NETSOFT.2015.7116120.
[14]	TESSLER C, MANKOWITZ J D, and MANNOR S. Reward constrained policy optimization[C]. International Conference on Learning Representations, New Orleans, USA, 2019. doi: 10.48550/arXiv.1805.11074.
[15]	MIJUMBI R, SERRAT J, GORRICHO J, et al. Network function virtualization: State-of-the-art and research challenges[J]. IEEE Communications Surveys & Tutorials, 2016, 18(1): 236–262. doi: 10.1109/COMST.2015.2477041.
[16]	ZHANG Yuchao, XU Ke, WANG Haiyang, et al. Going fast and fair: Latency optimization for cloud-based service chains[J]. IEEE Network, 2018, 32(2): 138–143. doi: 10.1109/MNET.2017.1700275.
[17]	WILLIAMS R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning[J]. Machine Learning, 1992, 8(3/4): 229–256. doi: 10.1007/BF00992696.
[18]	SUTTON R S and BARTO A G. Reinforcement Learning: An Introduction[M]. Cambridge, USA: MIT Press, 1998: 229–235.