Advanced Search
Volume 45 Issue 8
Aug.  2023
Turn off MathJax
Article Contents
TANG Lun, LI Shirui, DU Yucong, CHEN Qianbin. Deployment Algorithm of Service Function Chain Based on Multi-Agent Soft Actor-Critic Learning[J]. Journal of Electronics & Information Technology, 2023, 45(8): 2893-2901. doi: 10.11999/JEIT220803
Citation: TANG Lun, LI Shirui, DU Yucong, CHEN Qianbin. Deployment Algorithm of Service Function Chain Based on Multi-Agent Soft Actor-Critic Learning[J]. Journal of Electronics & Information Technology, 2023, 45(8): 2893-2901. doi: 10.11999/JEIT220803

Deployment Algorithm of Service Function Chain Based on Multi-Agent Soft Actor-Critic Learning

doi: 10.11999/JEIT220803
Funds:  The National Natural Science Foundation of China (62071078), The Science and Technology Research Program of Chongqing Municipal Education Commission (KJZD-M201800601), Sichuan Science and Technology Program (2021YFQ0053)
  • Received Date: 2022-06-17
  • Rev Recd Date: 2022-10-13
  • Available Online: 2022-12-23
  • Publish Date: 2023-08-21
  • Considering the problem of Service Function Chain (SFC) deployment optimization caused by the dynamic change of service requests under the Network Function Virtualization (NFV) architecture, an SFC deployment optimization algorithm based on Multi-Agent Soft Actor-Critic (MASAC) learning is proposed. Firstly, the model of minimizing resource load penalty, SFC deployment cost and delay cost is established, which is constrained by SFC end-to-end delay and reservation threshold of network resource. Secondly, the stochastic optimization is transformed into a Markov Decision Process (MDP) to realize the dynamic deployment of SFC and the balanced scheduling of resources. The arrangement scheme according to services division for multiple decision makers is further proposed. At last, the Soft Actor-Critic (SAC) algorithm is adopted in distributed multi-agent system to enhance exploration, then the central attention mechanism and advantage function are further introduced, which can dynamically and selectively focus on the information to obtain greater deployment return. Simulation results show that the proposed algorithm can optimize the load penalty, delay and deployment cost, and scale better with the increase of service requests.
  • loading
  • [1]
    CHAHBAR M, DIAZ G, DANDOUSH A, et al. A comprehensive survey on the E2E 5G network slicing model[J]. IEEE Transactions on Network and Service Management, 2021, 18(1): 49–62. doi: 10.1109/TNSM.2020.3044626
    [2]
    GONZALEZ A J, NENCIONI G, KAMISINSKI A, et al. Dependability of the NFV orchestrator: state of the art and research challenges[J]. IEEE Communications Surveys & Tutorials, 2018, 20(4): 3307–3329. doi: 10.1109/COMST.2018.2830648
    [3]
    SUN Gang, XU Zhu, YU Hongfang, et al. Low-latency and resource-efficient service function chaining orchestration in network function virtualization[J]. IEEE Internet of Things Journal, 2020, 7(7): 5760–5772. doi: 10.1109/JIOT.2019.2937110
    [4]
    LI Junling, SHI Weisen, YE Qiang, et al. Joint virtual network topology design and embedding for cybertwin-enabled 6G core networks[J]. IEEE Internet of Things Journal, 2021, 8(22): 16313–16325. doi: 10.1109/JIOT.2021.3097053
    [5]
    CHAI Rong, XIE Desheng, LUO Lei, et al. Multi-objective optimization-based virtual network embedding algorithm for software-defined networking[J]. IEEE Transactions on Network and Service Management, 2020, 17(1): 532–546. doi: 10.1109/TNSM.2019.2953297
    [6]
    CAO Haotong, DU Jianbo, ZHAO Haitao, et al. Resource-ability assisted service function chain embedding and scheduling for 6G networks with virtualization[J]. IEEE Transactions on Vehicular Technology, 2021, 70(4): 3846–3859. doi: 10.1109/TVT.2021.3065967
    [7]
    SOLOZABAL R, CEBERIO J, SANCHOYERTO A, et al. Virtual network function placement optimization with deep reinforcement learning[J]. IEEE Journal on Selected Areas in Communications, 2020, 38(2): 292–303. doi: 10.1109/JSAC.2019.2959183
    [8]
    CHEN Jing, CHEN Jia, and ZHANG Hongke. DRL-QOR: Deep reinforcement learning-based QoS/QoE-aware adaptive online orchestration in NFV-enabled networks[J]. IEEE Transactions on Network and Service Management, 2021, 18(2): 1758–1774. doi: 10.1109/TNSM.2021.3055494
    [9]
    HUANG Haojun, ZENG Cheng, ZHAO Yangmin, et al. Scalable orchestration of service function chains in NFV-enabled networks: A federated reinforcement learning approach[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(8): 2558–2571. doi: 10.1109/JSAC.2021.3087227
    [10]
    GHARBAOUI M, CONTOLI C, DAVOLI G, et al. Demonstration of latency-aware and self-adaptive service chaining in 5G/SDN/NFV infrastructures[C]. 2018 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), Verona, Italy, 2018: 1–2.
    [11]
    LIU Yu, SHANG Xiaojun, and YANG Yuanyuan. Joint SFC deployment and resource management in heterogeneous edge for latency minimization[J]. IEEE Transactions on Parallel and Distributed Systems, 2021, 32(8): 2131–2143. doi: 10.1109/TPDS.2021.3062341
    [12]
    YANG Jian, ZHANG Shuben, WU Xiaomin, et al. Online learning-based server provisioning for electricity cost reduction in data center[J]. IEEE Transactions on Control Systems Technology, 2017, 25(3): 1044–1051. doi: 10.1109/TCST.2016.2575801
    [13]
    PEI Jianing, HONG Peilin, XUE Kaiping, et al. Resource aware routing for service function chains in SDN and NFV-enabled network[J]. IEEE Transactions on Services Computing, 2021, 14(4): 985–997. doi: 10.1109/TSC.2018.2849712
    [14]
    VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. The 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 6000–6010.
    [15]
    LI Han, LÜ Tiejun, and ZHANG Xuewei. Deep deterministic policy gradient based dynamic power control for self-powered ultra-dense networks[C]. 2018 IEEE Globecom Workshops (GC Wkshps), Abu Dhabi, United Arab Emirates, 2018: 1–6.
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(8)  / Tables(1)

    Article Metrics

    Article views (437) PDF downloads(92) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return