高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于强化学习的非正交多址接入和移动边缘计算联合系统信息年龄更新

李保罡 石泰 陈静 李诗璐 王宇 张天魁

李保罡, 石泰, 陈静, 李诗璐, 王宇, 张天魁. 基于强化学习的非正交多址接入和移动边缘计算联合系统信息年龄更新[J]. 电子与信息学报, 2022, 44(12): 4238-4245. doi: 10.11999/JEIT211021
引用本文: 李保罡, 石泰, 陈静, 李诗璐, 王宇, 张天魁. 基于强化学习的非正交多址接入和移动边缘计算联合系统信息年龄更新[J]. 电子与信息学报, 2022, 44(12): 4238-4245. doi: 10.11999/JEIT211021
LI Baogang, SHI Tai, CHEN Jing, LI Shilu, WANG Yu, ZHANG Tiankui. Age of Information Updates in Non-Orthogonal Multiple Access-mobile Edge Computing System Based on Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2022, 44(12): 4238-4245. doi: 10.11999/JEIT211021
Citation: LI Baogang, SHI Tai, CHEN Jing, LI Shilu, WANG Yu, ZHANG Tiankui. Age of Information Updates in Non-Orthogonal Multiple Access-mobile Edge Computing System Based on Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2022, 44(12): 4238-4245. doi: 10.11999/JEIT211021

基于强化学习的非正交多址接入和移动边缘计算联合系统信息年龄更新

doi: 10.11999/JEIT211021
基金项目: 国家自然科学基金 (61971190),中央高校基本科研业务费专项资金项目(2019MS089),北京市自然科学基金(4164101),河北省高等学校科学技术研究项目(ZD2021406)
详细信息
    作者简介:

    李保罡:男,副教授,研究方向为无线通信、工业互联网、能源互联网、大数据分析等

    石泰:男,硕士生,研究方向为边缘计算、物理层安全等

    陈静:女,助理工程师,研究方向为互联网部运营监测

    李诗璐:女,硕士,研究方向为边缘计算

    王宇:男,硕士生,研究方向为物理层安全

    张天魁:男,教授,研究方向为移动边缘计算、无线资源管理、未来网络融合与管理等

    通讯作者:

    石泰 taishist@163.com

  • 中图分类号: TN91

Age of Information Updates in Non-Orthogonal Multiple Access-mobile Edge Computing System Based on Reinforcement Learning

Funds: The National Natural Science Foundation of China (61971190), The Fundamental Research Funds For the Central Universities (2019MS089), The Natural Science Foundation of Beijing (4164101), The Science and Technology Research Project of Colleges and Universities in Hebei (ZD2021406)
  • 摘要: 物联网发展对信息时效性的需求越来越高,信息新鲜度变得至关重要。为了维持信息新鲜度,在非正交多址接入(NOMA)和移动边缘计算(MEC)的联合系统中,对多设备单边缘计算服务器的传输场景进行了研究。在该场景中,如何分配卸载任务量和卸载功率以最小化平均更新代价是一个具有挑战性的问题。该文考虑到现实中的信道状态变化情况,基于多代理深度确定性策略梯度(MADDPG)算法,考虑信息新鲜度影响,建立了最小化平均更新代价的优化问题,提出一种寻找最优的卸载因子和卸载功率决策。仿真结果表明,采用部分卸载的方式可以有效地降低平均更新代价,利用MADDPG算法可以进一步优化卸载功率,经比较,MADDPG算法在降低平均更新代价方面优于其他方案,并且适当地减少设备数量在降低平均更新代价方面效果更好。
  • 图  1  在多个设备中数据的安全传输

    图  2  信息年龄的演变

    图  3  计算容量对平均更新代价的影响

    图  4  迭代次数和平均更新代价的关系

    图  5  不同设备数量对平均更新代价的影响

    表  1  仿真参数设置

    CPU周期信道带宽计算容量CPU频率干扰功率的代价总干扰功率
    $ 2 \times {10^3} $cycle/bit2 MHz10 GHz0.2 GHz0.120 W
    下载: 导出CSV
  • [1] KAUL S, YATES R, and GRUTESER M. Real-time status: How often should one update?[C]. 2012 IEEE INFOCOM, Orlando, USA, 2012: 2731–2735.
    [2] ZOU Peng, OZEL O, and SUBRAMANIAM S. Trading off computation with transmission in status update systems[C]. The IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Istanbul, Turkey, 2019: 1–6.
    [3] MAO Yuyi, YOU Changsheng, ZHANG Jun, et al. A survey on mobile edge computing: The communication perspective[J]. IEEE Communications Surveys & Tutorials, 2017, 19(4): 2322–2358. doi: 10.1109/COMST.2017.2745201
    [4] LI Rui, MA Qian, GONG Jie, et al. Age of processing: Age-driven status sampling and processing offloading for edge-computing-enabled real-time IoT applications[J]. IEEE Internet of Things Journal, 2021, 8(19): 14471–14484. doi: 10.1109/JIOT.2021.3064055
    [5] LIU Long, QIN Xiaoqi, TAO Yunzheng, et al. Timely updates in MEC-assisted status update systems: Joint task generation and computation offloading scheme[J]. China Communications, 2020, 17(8): 168–186. doi: 10.23919/JCC.2020.08.014
    [6] SONG Xianxin, QIN Xiaoqi, TAO Yunzheng, et al. Age based task scheduling and computation offloading in mobile-edge computing systems[C]. 2019 IEEE Wireless Communications and Networking Conference Workshop (WCNCW), Marrakech, Morocco, 2019: 1–6.
    [7] ZENG Jie, LV Tiejun, LIU Renping, et al. Investigation on evolving single-carrier NOMA into multi-carrier NOMA in 5G[J]. IEEE Access, 2018, 6: 48268–48288. doi: 10.1109/ACCESS.2018.2868093
    [8] MAATOUK A, ASSAAD M, and EPHREMIDES A. Minimizing the age of information: NOMA or OMA?[C]. 2019 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Paris, France, 2019: 102–108.
    [9] WANG Qian, CHEN He, LI Yonghui, et al. Minimizing age of information via hybrid NOMA/OMA[C]. 2020 IEEE International Symposium on Information Theory (ISIT), Los Angeles, USA, 2020: 1753–1758.
    [10] CHEN Xiang, GONG Fengkui, LI Guo, et al. User pairing and pair scheduling in massive MIMO-NOMA systems[J]. IEEE Communications Letters, 2018, 22(4): 788–791. doi: 10.1109/LCOMM.2017.2776206
    [11] DING Zhiguo, LIU Yuanwei, CHOI J, et al. Application of non-orthogonal multiple access in LTE and 5G networks[J]. IEEE Communications Magazine, 2017, 55(2): 185–191. doi: 10.1109/MCOM.2017.1500657CM
    [12] SAITO Y, KISHIYAMA Y, BENJEBBOUR A, et al. Non-Orthogonal Multiple Access (NOMA) for cellular future radio access[C]. The IEEE 77th Vehicular Technology Conference (VTC Spring), Dresden, Germany, 2013: 1–5.
    [13] ZHOU Bo and SAAD W. Joint status sampling and updating for minimizing age of information in the internet of things[J]. IEEE Transactions on Communications, 2019, 67(11): 7468–7482. doi: 10.1109/TCOMM.2019.2931538
    [14] YU Yuehua, CHEN He, LI Yonghui, et al. On the performance of non-orthogonal multiple access in short-packet communications[J]. IEEE Communications Letters, 2018, 22(3): 590–593. doi: 10.1109/LCOMM.2017.2786252
    [15] PAN Haoyuan, LIANG Jiaxin, LIEW S C, et al. Timely information update with nonorthogonal multiple access[J]. IEEE Transactions on Industrial Informatics, 2021, 17(6): 4096–4106. doi: 10.1109/TII.2020.3006061
    [16] GÓMEZ J T, MORALES-CÉSPEDES M, ARMADA A G, et al. Minimizing age of information on NOMA communication schemes for vehicular communication applications[C]. The 12th International Symposium on Communication Systems, Networks and Digital Signal Processing (CSNDSP), Porto, Portugal, 2020: 1–6.
    [17] ELGABLI A, KHAN H, KROUKA M, et al. Reinforcement learning based scheduling algorithm for optimizing age of information in Ultra Reliable low latency networks[C]. 2019 IEEE Symposium on Computers and Communications (ISCC), Barcelona, Spain, 2019: 1–6.
  • 加载中
图(5) / 表(1)
计量
  • 文章访问数:  363
  • HTML全文浏览量:  219
  • PDF下载量:  95
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-09-26
  • 修回日期:  2021-12-16
  • 录用日期:  2021-12-29
  • 网络出版日期:  2022-01-13
  • 刊出日期:  2022-12-10

目录

    /

    返回文章
    返回