基于深度强化学习的RIS辅助通感融合网络：挑战与机遇

陈真; 杜晓宇; 唐杰; WONGKat-Kit

doi:10.11999/JEIT240086

基于深度强化学习的RIS辅助通感融合网络：挑战与机遇

doi: 10.11999/JEIT240086

1.
香港城市大学电子工程系香港 999077
2.
华南理工大学电子与信息学院广州 510000
3.
伦敦大学学院电子与电气工程系伦敦(英国) W128LP

基金项目: 国家自然科学基金(62371197)，广东省自然科学基金(2022A1515011189)，东南大学开放课题(K202411)

详细信息

作者简介:
陈真：男，副研究员，研究方向为智能反射面通信、信道估计及波束管理

杜晓宇：女，硕士生，研究方向为智能反射面信道估计

唐杰：男，教授，研究方向为无线携能通信、智能反射面、绿色通信等

WONGKat-Kit：男，教授，研究方向为5G, 6G和流体天线系统等

通讯作者:
陈真　chenz.scut@gmail.com

中图分类号: TN929.5
计量
- 文章访问数: 608
- HTML全文浏览量: 223
- PDF下载量: 126
- 被引次数: 0
出版历程
- 收稿日期: 2024-02-22
- 修回日期: 2024-08-13
- 网络出版日期: 2024-08-27
- 刊出日期: 2024-09-26

DRL-based RIS-assisted ISAC Network: Challenges and Opportunities

1.
Department of Electronic Engineering, City University of Hong Kong, Hong Kong 999077 , China
2.
School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510000, China
3.
Department of Electronic and Electrical Engineering, University College London, London W128LP, UK

Funds: The National Natural Science Foundation of China (62371197), The National Natural Science Foundation of Guangdong (2022A1515011189), The Open Project of Southeast University (K202411)

摘要

摘要: 随着深度强化学习(DRL)技术的广泛应用，基于DRL的可重构智能表面(RIS)辅助的通信感知一体化(ISAC)展现出巨大的潜力。然而，由于数据卸载和模型训练的高成本，基于现有ISAC框架实现网络智能仍面临着巨大的挑战。为了克服该问题，该文深入分析了DRL技术在ISAC领域的应用，探讨了RIS辅助的ISAC建模及其解决方案，该技术能够解决覆盖区域受限、算法复杂度高以及高频传输等问题。为了推动这些技术的实施，该文进一步讨论了RIS辅助ISAC网络中DRL技术的未来发展趋势，包括潜在的应用和需要解决的问题。
- 深度强化学习 /
- 可重构智能表面 /
- 通信感知一体化
Abstract: The Deep Reinforcement Learning (DRL) has received widespread attention, which has potential in Reconfigurable Intelligent Surface (RIS) assisted Integrated Sensing And Communication (ISAC) network. However, due to the high cost of data offloading and model training, the existing RIS-assisted ISAC frameworks still face great challenges. To overcome this problem, the paper analyzes the main technology of DRL in the field of ISAC networks and its solution, which can solve the of high complexity, high-frequency transmission and limited coverage problems. To promote the implementation of these technologies, this paper further discusses the future development trends of DRL technologies in RIS-assisted ISAC networks, including potential applications and problems to be solved.
- Deep Reinforcement Learning (DRL) /
- Reconfigurable Intelligent Surface (RIS) /
- Integrated Sensing And Communication (ISAC)

HTML全文

图 1 RIS辅助ISAC系统的应用演示场景

下载: 全尺寸图片幻灯片

图 2 RIS辅助ISAC系统的DDPG设计

下载: 全尺寸图片幻灯片

图 3 RIS辅助ISAC系统的算力网络

下载: 全尺寸图片幻灯片

图 4 损失函数值与迭代次数的关系

下载: 全尺寸图片幻灯片

图 5 不同ISAC方案雷达探测率与迭代次数的关系

下载: 全尺寸图片幻灯片

表 1 基于深度学习的RIS辅助通信的最新进展

优化指标	RIS指标	DRL指标	场景	技术	结果
最大化保密率^[9]	相移控制波束赋形设计	参数设计	BS到RIS 多用户	DQN MDP	提高保密率
最大化加权和速率^[11]	相移控制波束赋形设计	参数设计	BS到RIS 多用户	比较检索法 DQNN	提高加权和速率
提高频谱效率^[14]	相移控制	互信息优化	BS到用户	DDPG TD3	提高频谱效率
最大化和速率^[15]	相移控制波束赋形设计	环境学习	RIS到用户多用户	NOMA协议 DRSAC	提高和速率
最大化总保密率^[16]	相移控制波束赋形设计	环境学习参数设计	多用户多窃听者	DDPG MDP	提高总保密率

下载: 导出CSV

表 2 参数设置

参数名称	参数值
RIS个数	10
训练大小	1000
训练学习率	0.001
迭代次数	20
测试集样本个数	500

下载: 导出CSV

参考文献(18)

[1]	LIU An, HUANG Zhe, LI Min, et al. A survey on fundamental limits of integrated sensing and communication[J]. IEEE Communications Surveys & Tutorials, 2022, 24(2): 994–1034. doi: 10.1109/COMST.2022.3149272.
[2]	CHEN Zhen, TANG Jie, HUANG Lei, et al. Robust target positioning for reconfigurable intelligent surface assisted MIMO radar systems[J]. IEEE Transactions on Vehicular Technology, 2023, 72(11): 15098–15102. doi: 10.1109/TVT.2023.3284454.
[3]	MEALEY R M. A method for calculating error probabilities in a radar communication system[J]. IEEE Transactions on Space Electronics and Telemetry, 1963, 9(2): 37–42. doi: 10.1109/TSET.1963.4337601.
[4]	ZHANG J A, RAHMAN M L, WU Kai, et al. Enabling joint communication and radar sensing in mobile networks—a survey[J]. IEEE Communications Surveys & Tutorials, 2022, 24(1): 306–345. doi: 10.1109/COMST.2021.3122519.
[5]	TONEX. Introduction to 6G \| IMT-2030[EB/OL]. https://www.tonex.com/training-courses/introduction-to-6g-imt-2030/, 2020.
[6]	CHEN Zhen, HUANG Lei, XIA Shuqiang, et al. Parallel channel estimation for RIS-assisted internet of things[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(8): 9762–9773. doi: 10.1109/TITS.2024.3364248.
[7]	Communication Network., ZTE and China Unicom Achieve World's First 5G Mid-Band Network Verification of Reconfigurable Intelligent Surface in External Networks[EB/OL] https://www.c114.com.cn/news/127/a1167167.html, 2021. Communication Network.,ZTE and China Unicom Achieve World's First 5G Mid-Band Network Verification of Reconfigurable Intelligent Surface in External Networks[EB/OL] https://www.c114.com.cn/news/127/a1167167.html, 2021.
[8]	CHEN Zhen, TANG Jie, ZHANG Xiuyin, et al. Hybrid evolutionary-based sparse channel estimation for IRS-assisted mmWave MIMO systems[J]. IEEE Transactions on Wireless Communications, 2022, 21(3): 1586–1601. doi: 10.1109/TWC.2021.3105405.
[9]	HUANG Chongwen, MO Ronghong, and YUEN C. Reconfigurable intelligent surface assisted multiuser MISO systems exploiting deep reinforcement learning[J]. IEEE Journal on Selected Areas in Communications, 2020, 38(8): 1839–1850. doi: 10.1109/JSAC.2020.3000835.
[10]	XU Wangyang, AN Jiancheng, XU Yongjun, et al. Time-varying channel prediction for RIS-assisted MU-MISO networks via deep learning[J]. IEEE Transactions on Cognitive Communications and Networking, 2022, 8(4): 1802–1815. doi: 10.1109/TCCN.2022.3188153.
[11]	YANG Helin, XIONG Zehui, ZHAO Jun, et al. Deep reinforcement learning-based intelligent reflecting surface for secure wireless communications[J]. IEEE Transactions on Wireless Communications, 2021, 20(1): 375–388. doi: 10.1109/TWC.2020.3024860.
[12]	XU Wangyang, GAN Lu, and HUANG Chongwen. A robust deep learning-based beamforming design for RIS-assisted multiuser MISO communications with practical constraints[J]. IEEE transactions on Cognitive Communications and Networking, 2022, 8(2): 694–706. doi: 10.1109/TCCN.2021.3128605.
[13]	DEMIR Ö T and BJÖRNSON E. Is channel estimation necessary to select phase-shifts for RIS-assisted massive MIMO?[J]. IEEE Transactions on Wireless Communications, 2022, 21(11): 9537–9552. doi: 10.1109/TWC.2022.3177700.
[14]	SAIKIA P, SINGH K, TAGHIZADEH O, et al. DRL algorithms for efficient spectrum sharing in RIS-aided MIMO radar and cellular systems[C]. MILCOM 2022 - 2022 IEEE Military Communications Conference (MILCOM), Rockville, USA, 2022: 55–60. doi: 10.1109/MILCOM55135.2022.10017985.
[15]	ZHAO Jingjing, YU Lanchenhui, CAI Kaiquan, et al. RIS-aided ground-aerial NOMA communications: A distributionally robust DRL approach[J]. IEEE Journal on Selected Areas in Communications, 2022, 40(4): 1287–1301. doi: 10.1109/JSAC.2022.3143230.
[16]	PENG Zhangjie, ZHANG Zhibo, KONG Lei, et al. Deep reinforcement learning for RIS-aided multiuser full-duplex secure communications with hardware impairments[J]. IEEE Internet of Things Journal, 2022, 9(21): 21121–21135. doi: 10.1109/JIOT.2022.3177705.
[17]	张在琛, 江浩. 智能超表面使能无人机高能效通信信道建模与传输机理分析[J]. 电子学报, 2023, 51(10): 2623–2634. doi: 10.12263/DZXB.20221352. ZHANG Zaichen and JIANG Hao. Channel modeling and characteristics analysis for high energy-efficient RIS-assisted UAV communications[J]. Acta Electronica Sinica, 2023, 51(10): 2623–2634. doi: 10.12263/DZXB.20221352.
[18]	ZHOU Hao, EROL-KANTARCI M, LIU Yuanwei, et al. Heuristic algorithms for RIS-assisted wireless networks: Exploring heuristic-aided machine learning[J]. IEEE Wireless Communications, 2024, 31(4): 106–114. doi: 10.1109/MWC.010.2300321.