Online Service Function Chain Deployment Method Based on Deep Q Network

Hang QIU; Hongbo TANG; Wei YOU

doi:10.11999/JEIT201009

Volume 43 Issue 11

Nov. 2021

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2021 > 43(11): 3122-3130

Hang QIU, Hongbo TANG, Wei YOU. Online Service Function Chain Deployment Method Based on Deep Q Network[J]. Journal of Electronics & Information Technology, 2021, 43(11): 3122-3130. doi: 10.11999/JEIT201009

Citation:

Hang QIU, Hongbo TANG, Wei YOU. Online Service Function Chain Deployment Method Based on Deep Q Network[J]. Journal of Electronics & Information Technology, 2021, 43(11): 3122-3130. doi: 10.11999/JEIT201009

Citation:

PDF( 1926 KB)

Online Service Function Chain Deployment Method Based on Deep Q Network

doi: 10.11999/JEIT201009

Information Engineering University People’s Liberation Army Strategic Support Force, Zhengzhou 450002, China

Funds: The National Natural Science Foundation of China (61801515, 61941114, 61521003)

Received Date: 2020-12-02
Rev Recd Date: 2021-06-30

Available Online: 2021-08-10

Publish Date: 2021-11-23

Abstract

Abstract

To deal with the dynamic nature of 5G network resource state and the difficulty of service function chain deployment under the high-dimensional network state model, an online Service function Chain Deployment method based on Deep Q network (DeePSCD) is proposed. First, to describe the dynamic nature of network resource state, the service function chain deployment is modeled as a Markov decision process. Then, the deep Q network is used to solve the online service function chain deployment problem in the high-dimensional system resource model. This method can effectively describe the dynamic changes of network resource state. Specifically, deep Q network handles the complexity of problem and determines the optimal deployment solution of service function chain. Simulation results show that the proposed method can reduce the deployment cost of the service function chain and increase the acceptance rate while meeting the service delay constraint.
- Network function virtualization,
- Service function chain,
- Markov decision process,
- Deep Q network

FullText(HTML)

References(26)

References

[1]	ETSI. Network Functions Virtualisation (NFV)[EB/OL]. https://www.etsi.org/technologies/nfv, 2020.
[2]	YI Bo, WANG Xingwei, LI Keqin, et al. A comprehensive survey of network function virtualization[J]. Computer Networks, 2018, 133: 212–262. doi: 10.1016/j.comnet.2018.01.021
[3]	ERAMO V, MIUCCI E, AMMAR M, et al. An approach for service function chain routing and virtual function network instance migration in network function virtualization architectures[J]. IEEE/ACM Transactions on Networking, 2017, 25(4): 2008–2025. doi: 10.1109/TNET.2017.2668470
[4]	KARAKUS M and DURRESI A. A survey: Control plane scalability issues and approaches in Software-Defined Networking (SDN)[J]. Computer Networks, 2017, 112: 279–293. doi: 10.1016/j.comnet.2016.11.017
[5]	BHAMARE D, JAIN R, SAMAKA M, et al. A survey on service function chaining[J]. Journal of Network and Computer Applications, 2016, 75: 138–155. doi: 10.1016/j.jnca.2016.09.001
[6]	MIJUMBI R, SERRAT J, GORRICHO J L, et al. Network function virtualization: State-of-the-art and research challenges[J]. IEEE Communications Surveys & Tutorials, 2016, 18(1): 236–262. doi: 10.1109/COMST.2015.2477041
[7]	BARI M F, CHOWDHURY S R, AHMED R, et al. Orchestrating virtualized network functions[J]. IEEE Transactions on Network and Service Management, 2016, 13(4): 725–739. doi: 10.1109/TNSM.2016.2569020
[8]	LIU Jiaqiang, LI Yong, ZHANG Ying, et al. Improve service chaining performance with optimized middlebox placement[J]. IEEE Transactions on Services Computing, 2017, 10(4): 560–573. doi: 10.1109/TSC.2015.2502252
[9]	KUO T W, LIOU B H, LIN K C J, et al. Deploying chains of virtual network functions: On the relation between link and server usage[C]. IEEE INFOCOM 2016 - the 35th Annual IEEE International Conference on Computer Communications, San Francisco, USA, 2016: 1–9. doi: 10.1109/INFOCOM.2016.7524565.
[10]	SUN Quanying, LU Ping, LU Wei, et al. Forecast-assisted NFV service chain deployment based on affiliation-aware vNF placement[C]. 2016 IEEE Global Communications Conference (GLOBECOM), Washington, USA, 2016: 1–6. doi: 10.1109/GLOCOM.2016.7841846.
[11]	LI Defang, HONG Peilin, XUE Kaiping, et al. Virtual network function placement considering resource optimization and SFC requests in cloud datacenter[J]. IEEE Transactions on Parallel and Distributed Systems, 2018, 29(7): 1664–1677. doi: 10.1109/TPDS.2018.2802518
[12]	HAWILO H, JAMMAL M, and SHAMI A. Network function virtualization-aware orchestrator for service function chaining placement in the cloud[J]. IEEE Journal on Selected Areas in Communications, 2019, 37(3): 643–655. doi: 10.1109/JSAC.2019.2895226
[13]	TANG Hong, ZHOU D, and CHEN Duan. Dynamic network function instance scaling based on traffic forecasting and VNF placement in operator data centers[J]. IEEE Transactions on Parallel and Distributed Systems, 2019, 30(3): 530–543. doi: 10.1109/TPDS.2018.2867587
[14]	QI Dandan, SHEN Subin, and WANG Guanghui. Towards an efficient VNF placement in network function virtualization[J]. Computer Communications, 2019, 138: 81–89. doi: 10.1016/j.comcom.2019.03.005
[15]	SINGH S, OKUN A, and JACKSON A. Learning to play Go from scratch[J]. Nature, 2017, 550(7676): 336–337. doi: 10.1038/550336a
[16]	袁泉, 汤红波, 黄开枝, 等. 基于Q-learning算法的vEPC虚拟网络功能部署方法[J]. 通信学报, 2017, 38(8): 172–182. doi: 10.11959/j.issn.1000-436x.2017173 YUAN Quan, TANG Hongbo, HUANG Kaizhi, et al. Deployment method for vEPC virtualized network function via Q-learning[J]. Journal on Communications, 2017, 38(8): 172–182. doi: 10.11959/j.issn.1000-436x.2017173
[17]	XIAO Yikai, ZHANG Qixia, LIU Fangming, et al. NFVdeep: Adaptive online service function chain deployment with deep reinforcement learning[C]. Proceedings of the International Symposium on Quality of Service, Arizona, USA, 2019: 1–10. doi: 10.1145/3326285.3329056.
[18]	QUANG P T A, HADJADJ-AOUL Y, and OUTTAGARTS A. A deep reinforcement learning approach for VNF forwarding graph embedding[J]. IEEE Transactions on Network and Service Management, 2019, 16(4): 1318–1331. doi: 10.1109/TNSM.2019.2947905
[19]	KINGMAN J F C. Poisson Processes[M]. ARMITAGE P and COLTON T. Encyclopedia of Biostatistics. Chichester: John Wiley & Sons, 2005. doi: 10.1002/0470011815.b2a07042.
[20]	HOWARD R A. Dynamic programming and Markov processes[J]. Technometrics, 1961, 3(1): 120–121. doi: 10.2307/1266484
[21]	The University of Adelaide. The internet topology zoo[EB/OL]. http://www.topology-zoo.org/dataset.html, 2012.
[22]	Networkx. Network analysis in python: Important structures and bipartite graphs[EB/OL]. https://coderzcolumn.com/tutorials/data-science/network-analysis-in-python-important-structures-and-bipartite-graphs-networkx, 2020.
[23]	YALA L, FRANGOUDIS P A, LUCARELLI G, et al. Cost and availability aware resource allocation and virtual function placement for CDNaaS provision[J]. IEEE Transactions on Network and Service Management, 2018, 15(4): 1334–1348. doi: 10.1109/TNSM.2018.2874524
[24]	OCHOA-ADAY L, CERVELLÓ-PASTOR C, FERNÁNDEZ-FERNÁNDEZ A, et al. An online algorithm for dynamic NFV placement in cloud-based autonomous response networks[J]. Symmetry, 2018, 10(5): 163. doi: 10.3390/sym10050163
[25]	MAROTTA A, ZOLA E, D’ANDREAGIOVANNI F, et al. A fast robust optimization-based heuristic for the deployment of green virtual network functions[J]. Journal of Network and Computer Applications, 2017, 95: 42–53. doi: 10.1016/j.jnca.2017.07.014
[26]	SHI Runyu, ZHANG Jia, CHU Wenjing, et al. MDP and machine learning-based cost-optimization of dynamic resource allocation for network function virtualization[C]. 2015 IEEE International Conference on Services Computing, New York, USA, 2015: 65–73. doi: 10.1109/SCC.2015.19.