Physical Layer Security Algorithm of Reconfigurable Intelligent Surface-assisted Unmanned Aerial Vehicle Communication System Based on Reinforcement Learning

HU Langtao; BI Songjiao; LIU Quanjin; WU Jianlan; YANG Rui; WANG Hong

doi:10.11999/JEIT211613

Volume 44 Issue 7

Jul. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(7): 2407-2415

HU Langtao, BI Songjiao, LIU Quanjin, WU Jianlan, YANG Rui, WANG Hong. Physical Layer Security Algorithm of Reconfigurable Intelligent Surface-assisted Unmanned Aerial Vehicle Communication System Based on Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2022, 44(7): 2407-2415. doi: 10.11999/JEIT211613

Citation:

HU Langtao, BI Songjiao, LIU Quanjin, WU Jianlan, YANG Rui, WANG Hong. Physical Layer Security Algorithm of Reconfigurable Intelligent Surface-assisted Unmanned Aerial Vehicle Communication System Based on Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2022, 44(7): 2407-2415. doi: 10.11999/JEIT211613

Citation:

HU Langtao, BI Songjiao, LIU Quanjin, WU Jianlan, YANG Rui, WANG Hong. Physical Layer Security Algorithm of Reconfigurable Intelligent Surface-assisted Unmanned Aerial Vehicle Communication System Based on Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2022, 44(7): 2407-2415. doi: 10.11999/JEIT211613

PDF( 3054 KB)

Physical Layer Security Algorithm of Reconfigurable Intelligent Surface-assisted Unmanned Aerial Vehicle Communication System Based on Reinforcement Learning

doi: 10.11999/JEIT211613 cstr: 32379.14.JEIT211613

1.
School of Electronic Engineering and Intelligent Manufacturing, Anqing Normal University, Anqing 246133, China
2.
Key Laboratory of Intelligent Perception and Computing in Anhui Province, Anqing 246133, China
3.
Anhui Province Railway Investment Co. LTD, Hefei 230601, China

Funds: The National Natural Science Foundation of China (62171002), The Natural Science Foundation of Anhui Provincial Department of Education (KJ2019A0554)

Received Date: 2021-12-24
Rev Recd Date: 2022-05-03

Available Online: 2022-05-08

Publish Date: 2022-07-25

Abstract

Abstract

In this paper, the optimization problem of the 3D trajectory for Unmanned Aerial Vehicle (UAV) assisted by Reconfigurable Intelligent Surface (RIS) in physical layer security is studied. Specifically, when the RIS assisted UAV transmits wirelessly information to the ground user, the physical layer security rate is maximized by jointly optimizing the RIS phase shift and the UAV's 3D trajectory. However, because the objective function is non convex, the traditional optimization technology is difficult to solve it directly. The dynamic and complex optimization problems in wireless communication can be solved by deep reinforcement learning. Based on reinforcement learning Double Deep Q Network (DDQN), a joint optimization algorithm of RIS phase shift and UAV 3D trajectory is designed in this paper to maximize the achievable average safety rate. The simulation results show that the designed RIS assisted UAV communication optimization algorithm can obtain higher safety rate than the Successive Convex Approximation (SCA) algorithm with fixed flight altitude, RIS algorithm with random phase shift and algorithm without RIS.
- Deep reinforcement learning,
- Reconfigurable Intelligent Surface(RIS),
- Unmanned Aerial Vehicle (UAV),
- Physical layer security

FullText(HTML)

References(27)

References

[1]	ZHOU Xiaobo, WU Qingqing, YAN Shihao, et al. UAV-enabled secure communications: Joint trajectory and transmit power optimization[J]. IEEE Transactions on Vehicular Technology, 2019, 68(4): 4069–4073. doi: 10.1109/TVT.2019.2900157
[2]	WU Qingqing, ZENG Yong, and ZHANG Rui. Joint trajectory and communication design for multi-UAV enabled wireless networks[J]. IEEE Transactions on Wireless Communications, 2018, 17(3): 2109–2121. doi: 10.1109/TWC.2017.2789293
[3]	ZENG Yong, ZHANG Rui, and LIM T J. Throughput maximization for UAV-enabled mobile relaying systems[J]. IEEE Transactions on Communications, 2016, 64(12): 4983–4996. doi: 10.1109/TCOMM.2016.2611512
[4]	ZHAO Nan, CHENG Fen, YU F R, et al. Caching UAV assisted secure transmission in hyper-dense networks based on interference alignment[J]. IEEE Transactions on Communications, 2018, 66(5): 2281–2294. doi: 10.1109/TCOMM.2018.2792014
[5]	ZHAN Cheng, ZENG Yong, and ZHANG Rui. Energy-efficient data collection in UAV enabled wireless sensor network[J]. IEEE Wireless Communications Letters, 2018, 7(3): 328–331. doi: 10.1109/LWC.2017.2776922
[6]	FU Yujing, MEI Haibo, WANG Kezhi, et al. Joint optimization of 3D trajectory and scheduling for solar-powered UAV systems[J]. IEEE Transactions on Vehicular Technology, 2021, 70(4): 3972–3977. doi: 10.1109/TVT.2021.3063310
[7]	YAN Shihao, ZHOU Xiangyun, YANG Nan, et al. Artificial-noise-aided secure transmission in wiretap channels with transmitter-side correlation[J]. IEEE Transactions on Wireless Communications, 2016, 15(12): 8286–8297. doi: 10.1109/TWC.2016.2613860
[8]	YAN Shihao, YANG Nan, LAND I, et al. Three artificial-noise-aided secure transmission schemes in wiretap channels[J]. IEEE Transactions on Vehicular Technology, 2018, 67(4): 3669–3673. doi: 10.1109/TVT.2017.2779508
[9]	ZHANG Guangchi, WU Qingqing, CUI Miao, et al. Securing UAV communications via joint trajectory and power control[J]. IEEE Transactions on Wireless Communications, 2019, 18(2): 1376–1389. doi: 10.1109/TWC.2019.2892461
[10]	LI An, WU Qingqing, and ZHANG Rui. UAV-enabled cooperative jamming for improving secrecy of ground wiretap channel[J]. IEEE Wireless Communications Letters, 2019, 8(1): 181–184. doi: 10.1109/LWC.2018.2865774
[11]	WU Qingqing, LIU Liang, and ZHANG Rui. Fundamental trade-offs in communication and trajectory design for UAV-enabled wireless network[J]. IEEE Wireless Communications, 2019, 26(1): 36–44. doi: 10.1109/MWC.2018.1800221
[12]	LI Sixian, DUO Bin, YUAN Xiaojun, et al. Reconfigurable intelligent surface assisted UAV communication: Joint trajectory design and passive beamforming[J]. IEEE Wireless Communications Letters, 2020, 9(5): 716–720. doi: 10.1109/LWC.2020.2966705
[13]	FANG Sisai, CHEN Gaojie, and LI Yonghui. Joint optimization for secure intelligent reflecting surface assisted UAV networks[J]. IEEE Wireless Communications Letters, 2021, 10(2): 276–280. doi: 10.1109/LWC.2020.3027969
[14]	FANG Junhao, YANG Zhaohui, ANJUM N, et al. Secure intelligent reflecting surface assisted UAV communication networks[C]. 2021 IEEE International Conference on Communications Workshops (ICC Workshops), Montreal, Canada, 2021.
[15]	陈新颖, 盛敏, 李博, 等. 面向6G的无人机通信综述[J]. 电子与信息学报, 2022, 44(3): 781–789. doi: 10.11999/JEIT210789 CHEN Xinying, SHENG Min, LI Bo, et al. Survey on unmanned aerial vehicle communications for 6G[J]. Journal of Electronics &Information Technology, 2022, 44(3): 781–789. doi: 10.11999/JEIT210789
[16]	XU Yongjun, XIE Hao, WU Qingqing, et al. Robust max-min energy efficiency for RIS-aided HetNets with distortion noises[J]. IEEE Transactions on Communications, 2022, 70(2): 1457–1471. doi: 10.1109/TCOMM.2022.3141798
[17]	XU Yongjun, GAO Zhengnian, WANG Zhengqiang, et al. RIS-enhanced WPCNs: Joint radio resource allocation and passive beamforming optimization[J]. IEEE Transactions on Vehicular Technology, 2021, 70(8): 7980–7991. doi: 10.1109/TVT.2021.3096603
[18]	ZHANG Jiayi, DU Hongyang, SUN Qiang, et al. Physical layer security enhancement with reconfigurable intelligent surface-aided networks[J]. IEEE Transactions on Information Forensics and Security, 2021, 16: 3480–3495. doi: 10.1109/TIFS.2021.3083409
[19]	HUANG Chongwen, ZAPPONE A, ALEXANDROPOULOS G C, et al. Reconfigurable intelligent surfaces for energy efficiency in wireless communication[J]. IEEE Transactions on Wireless Communications, 2019, 18(8): 4157–4170. doi: 10.1109/TWC.2019.2922609
[20]	HUANG Chongwen, MO Ronghong, and YUEN C. Reconfigurable intelligent surface assisted multiuser MISO systems exploiting deep reinforcement learning[J]. IEEE Journal on Selected Areas in Communications, 2020, 38(8): 1839–1850. doi: 10.1109/JSAC.2020.3000835
[21]	ZHANG Yu, ZHUANG Zirui, GAO Feifei, et al. Multi-agent deep reinforcement learning for secure UAV communications[C]. 2020 IEEE Wireless Communications and Networking Conference (WCNC), Seoul, Korea, 2020: 1–5.
[22]	FU Fang, JIAO Qi, YU F R, et al. Securing UAV-to-vehicle communications: A curiosity-driven deep Q-learning network (C-DQN) approach[C]. 2021 IEEE International Conference on Communications Workshops (ICC Workshops), Montreal, Canada, 2021.
[23]	ZHANG Yu, MOU Zhiyu, GAO Feifei, et al. UAV-enabled secure communications by multi-agent deep reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2020, 69(10): 11599–11611. doi: 10.1109/TVT.2020.3014788
[24]	MEI Haibo, YANG Kun, LIU Qiang, et al. 3D-trajectory and phase-shift design for RIS-assisted UAV systems using deep reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2022, 71(3): 3020–3029. doi: 10.1109/TVT.2022.3143839
[25]	WATKINS C J C H and DAYAN P. Q-learning[J]. Machine Learning, 1992, 8(3/4): 279–292. doi: 10.1007/BF00992698
[26]	MNIH V, KAVUKCUOGLU K, SILVER D, et al. Playing atari with deep reinforcement learning[EB/OL]. https://arxiv.org/abs/1312.5602, 2013.
[27]	NASIR Y S and GUO Dongning. Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks[J]. IEEE Journal on Selected Areas in Communications, 2019, 37(10): 2239–2250. doi: 10.1109/JSAC.2019.2933973