| Citation: | Lun TANG, Lanqin HE, Qi TAN, Qianbin CHEN. Virtual Network Function Migration Optimization Algorithm Based on Deep Deterministic Policy Gradient[J]. Journal of Electronics & Information Technology, 2021, 43(2): 404-411. doi: 10.11999/JEIT190921
|
To solve the problem of Virtual Network Function (VNF) migration optimization, which is caused by the dynamic change of resource requirements of Service Function Chain (SFC) under Network Function Virtualization/ Software Defined Network (NFV/SDN) architecture, a VNF migration optimization algorithm is proposed based on deep reinforcement learning. Firstly, based on the underlying CPU, bandwidth resources and SFC end-to-end delay constraints, a Markov Decision Process (MDP) based stochastic optimization model is established. This model is used to optimize jointly network energy consumption and SFC end-to-end delay by migrating VNF. Secondly, since the state space and action space of this paper are continuous value sets, a VNF intelligent migration algorithm based on Deep Deterministic Policy Gradient (DDPG) is proposed to obtain an approximate optimal VNF migration strategy. The simulation results show that the algorithm can achieve the compromise between network energy consumption and SFC end-to-end delay, and improve the resource utilization of the physical network.
|
唐伦, 周钰, 杨友超, 等. 5G网络切片场景中基于预测的虚拟网络功能动态部署算法[J]. 电子与信息学报, 2019, 41(9): 2071–2078. doi: 10.11999/JEIT180894
TANG Lun, ZHOU Yu, YANG Youchao, et al. Virtual network function dynamic deployment algorithm based on prediction for 5g network slicing[J]. Journal of Electronics &Information Technology, 2019, 41(9): 2071–2078. doi: 10.11999/JEIT180894
|
|
唐伦, 杨恒, 马润琳, 等. 基于5G接入网络的多优先级虚拟网络功能迁移开销与网络能耗联合优化算法[J]. 电子与信息学报, 2019, 41(9): 2079–2086. doi: 10.11999/JEIT180906
TANG Lun, YANG Heng, MA Runlin, et al. Multi-priority based joint optimization algorithm of virtual network function migration cost and network energy consumption[J]. Journal of Electronics &Information Technology, 2019, 41(9): 2079–2086. doi: 10.11999/JEIT180906
|
|
程国振. 基于元能力的网络功能组合关键技术研究[D]. [博士论文], 解放军信息工程大学, 2015.
CHENG Guozhen. Research on the key technologies of function composition based on atomic capability[D]. [Ph. D. dissertation], Information Engineering University, 2015.
|
|
CHO D, TAHERI J, ZOMAYA A Y, et al. Real-time virtual network function (VNF) migration toward low network latency in cloud environments[C]. The 10th IEEE International Conference on Cloud Computing (CLOUD), Honolulu, USA, 2017: 798–801. doi: 10.1109/CLOUD.2017.118.
|
|
XIA Jing, CAI Zhiping, and XU Ming. Optimized virtual network functions migration for NFV[C]. The 22nd IEEE International Conference on Parallel and Distributed Systems (ICPADS), Wuhan, China, 2016: 340–246. doi: 10.1109/ICPADS.2016.0053.
|
|
GHARBAOUI M, CONTOLI C, DAVOLI G, et al. Demonstration of latency-aware and self-adaptive service chaining in 5G/SDN/NFV infrastructures[C]. 2018 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), Verona, Italy, 2018: 1–2. doi: 10.1109/NFV-SDN.2018.8725645.
|
|
唐伦, 赵培培, 赵国繁, 等. 基于QoS保障的服务功能链动态部署算法[J]. 北京邮电大学学报, 2018, 41(6): 90–96. doi: 10.13190/j.jbupt.2018-013
TANG Lun, ZHAO Peipei, ZHAO Guofan, et al. Dynamic deployment algorithm for service function chaining with QoS guarantee[J]. Journal of Beijing University of Posts and Telecommunications, 2018, 41(6): 90–96. doi: 10.13190/j.jbupt.2018-013
|
|
ERAMO V, AMMAR M, and LAVACCA F G. Migration energy aware reconfigurations of virtual network function instances in NFV architectures[J]. IEEE Access, 2017, 5: 4927–4938. doi: 10.1109/ACCESS.2017.2685437
|
|
LI Han, GAO Hui, LÜ Tiejun, et al. Deep q-learning based dynamic resource allocation for self-powered ultra-dense networks[C]. 2018 IEEE International Conference on Communications Workshops (ICC Workshops), Kansas City, USA, 2018: 1–6. doi: 10.1109/ICCW.2018.8403505.
|
|
YE Junhong and ZHANG Y J A. DRAG: Deep reinforcement learning based base station activation in heterogeneous networks[J]. IEEE Transactions on Mobile Computing, 2020, 19(9): 2076–2087. doi: 10.1109/TMC.2019.2922602
|
|
CHU Man, LIAO Xuewen, LI Hang, et al. Power control in energy harvesting multiple access system with reinforcement learning[J]. IEEE Internet of Things Journal, 2019, 6(5): 9175–9186. doi: 10.1109/JIOT.2019.2928837
|
|
LI Han, LÜ Tiejun, and ZHANG Xuewei. Deep deterministic policy gradient based dynamic power control for self-powered ultra-dense networks[C]. 2018 IEEE Globecom Workshops (GC Wkshps), Abu Dhabi, United Arab Emirates, 2018: 1–6. doi: 10.1109/GLOCOMW.2018.8644157.
|
|
ERAMO V, MIUCCI E, AMMAR M, et al. An approach for service function chain routing and virtual function network instance migration in network function virtualization architectures[J]. IEEE/ACM Transactions on Networking, 2017, 25(4): 2008–2025. doi: 10.1109/TNET.2017.2668470
|
Figures(8) / Tables(2)
DownLoad:
