Energy Aware Reconfigurable Intelligent Surfaces Assisted Unmanned Aerial Vehicle Age of Information Enabled Data Collection Policies

ZHANG Tao; ZHANG Qian; ZHU Yingwen; DAI Chen

doi:10.11999/JEIT240866

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

ZHANG Tao, ZHANG Qian, ZHU Yingwen, DAI Chen. Energy Aware Reconfigurable Intelligent Surfaces Assisted Unmanned Aerial Vehicle Age of Information Enabled Data Collection Policies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240866

Citation:

ZHANG Tao, ZHANG Qian, ZHU Yingwen, DAI Chen. Energy Aware Reconfigurable Intelligent Surfaces Assisted Unmanned Aerial Vehicle Age of Information Enabled Data Collection Policies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240866

Citation:

ZHANG Tao, ZHANG Qian, ZHU Yingwen, DAI Chen. Energy Aware Reconfigurable Intelligent Surfaces Assisted Unmanned Aerial Vehicle Age of Information Enabled Data Collection Policies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240866

PDF( 2092 KB)

Energy Aware Reconfigurable Intelligent Surfaces Assisted Unmanned Aerial Vehicle Age of Information Enabled Data Collection Policies

doi: 10.11999/JEIT240866

ZHANG Tao^{1, 2},
ZHANG Qian^{1
,
,},
ZHU Yingwen¹,
DAI Chen³

1.
School of Information Technology, Jiangsu Open University, Nanjing 210000, China
2.
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China

Funds: The National Natural Science Foundation of China (62402232), The Natural Science Foundation of the Jiangsu Higher Education Institutions of China (23KJB520024)

Received Date: 2024-10-14
Rev Recd Date: 2025-01-07

Available Online: 2025-01-11

Abstract

Abstract

Objective : To address the balance between efficient energy utilization and information freshness in UAV-assisted data collection for the Internet of Things (IoT) using Reconfigurable Intelligent Surfaces (RIS). Methods : A data collection optimization policy based on deep reinforcement learning is proposed. Considering flight energy consumption, communication complexity, and Age of Information (AoI) constraints, a joint optimization scheme is designed using a Double Deep Q-Network (DDQN). The scheme integrates UAV trajectory planning, IoT device scheduling, and RIS phase adjustment, mitigating Q-value overestimation observed in traditional Q-learning methods. Results and Discussions : The proposed method enables the UAV to dynamically adjust its trajectory and communication strategy based on real-time environmental conditions, enhancing data transmission efficiency and reducing energy consumption. Simulation results demonstrate superior convergence compared with traditional methods (Fig. 3). The UAV trajectory shows that the proposed method effectively accomplishes the data collection task (Fig. 4). Furthermore, rational allocation of energy and communication resources allows dynamic adaptation to varying communication environment parameters, ensuring an optimal balance between energy consumption and AoI (Fig. 5)(Fig. 6). Conclusions : The deep reinforcement learning-based optimization policy for UAV-assisted IoT data collection with RIS effectively resolves the trade-off between energy utilization and information freshness. This robust solution improves data collection efficiency in dynamic communication environments.
- UAV-assisted communication,
- Age of Information (AoI),
- Deep reinforcement learning,
- Reconfigurable Intelligent Surfaces (RIS)

FullText(HTML)

References(34)

References

[1]	段洁, 胡显静, 林欢, 等. 面向物联网数据特征的信息中心网络缓存方案[J]. 电子与信息学报, 2021, 43(8): 2240–2248. doi: 10.11999/JEIT200631. DUAN Jie, HU Xianjing, LIN Huan, et al. Information-centric networking caching scheme for data characteristics of internet of things[J]. Journal of Electronics & Information Technology, 2021, 43(8): 2240–2248. doi: 10.11999/JEIT200631.
[2]	JAVAID S, SAEED N, QADIR Z, et al. Communication and control in collaborative UAVs: Recent advances and future trends[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(6): 5719–5739. doi: 10.1109/TITS.2023.3248841.
[3]	刘志新, 赵松晗, 杨毅, 等. 智能反射面辅助的无人机无线携能通信网络吞吐量最大化算法研究[J]. 电子与信息学报, 2022, 44(7): 2325–2331. doi: 10.11999/JEIT220195. LIU Zhixin, ZHAO Songhan, YANG Yi, et al. Throughput maximization algorithm for intelligent reflecting surface-aided unmanned aerial vehicle communication networks with wireless energy transfer[J]. Journal of Electronics & Information Technology, 2022, 44(7): 2325–2331. doi: 10.11999/JEIT220195.
[4]	张在琛, 江浩. 智能超表面使能无人机高能效通信信道建模与传输机理分析[J]. 电子学报, 2023, 51(10): 2623–2634. doi: 10.12263/DZXB.20221352. ZHANG Zaichen and JIANG Hao. Channel modeling and characteristics analysis for high energy-efficient RIS-assisted UAV communications[J]. Acta Electronica Sinica, 2023, 51(10): 2623–2634. doi: 10.12263/DZXB.20221352.
[5]	SAVKIN A V, HUANG Chao, and NI Wei. Joint multi-UAV path planning and LoS communication for mobile-edge computing in IoT networks with RISs[J]. IEEE Internet of Things Journal, 2023, 10(3): 2720–2727. doi: 10.1109/JIOT.2022.3215255.
[6]	SAVKIN A V, HUANG Chao, and NI Wei. Collision-free 3-D navigation of a UAV team for optimal data collection in Internet-of-Things networks with reconfigurable intelligent surfaces[J]. IEEE Systems Journal, 2023, 17(3): 4070–4077. doi: 10.1109/JSYST.2023.3269095.
[7]	LIN Xinzhong, XIE Cong, XIE Wenwu, et al. Security performance analysis of RIS-assisted UAV wireless communication in industrial IoT[J]. The Journal of Supercomputing, 2022, 78(4): 5957–5973. doi: 10.1007/s11227-021-04095-7.
[8]	ZHAI Liangsen, ZOU Yulong, ZHU Jia, et al. RIS-assisted UAV-enabled wireless powered communications: System modeling and optimization[J]. IEEE Transactions on Wireless Communications, 2024, 23(5): 5094–5108. doi: 10.1109/TWC.2023.3324500.
[9]	RANJHA A and KADDOUM G. URLLC facilitated by mobile UAV relay and RIS: A joint design of passive beamforming, blocklength, and UAV positioning[J]. IEEE Internet of Things Journal, 2021, 8(6): 4618–4627. doi: 10.1109/JIOT.2020.3027149.
[10]	ESKANDARI M, HUANG Hailong, SAVKIN A V, et al. Model predictive control-based 3D navigation of a RIS-equipped UAV for LoS wireless communication with a ground intelligent vehicle[J]. IEEE Transactions on Intelligent Vehicles, 2023, 8(3): 2371–2384. doi: 10.1109/TIV.2022.3232890.
[11]	LI Linpei, GUAN Wanqing, ZHAO Chuan, et al. Trajectory planning, phase shift design, and IoT devices association in flying-RIS-assisted mobile edge computing[J]. IEEE Internet of Things Journal, 2024, 11(1): 147–157. doi: 10.1109/JIOT.2023.3300700.
[12]	AL-HILO A, SAMIR M, ELHATTAB M, et al. RIS-assisted UAV for timely data collection in IoT networks[J]. IEEE Systems Journal, 2023, 17(1): 431–442. doi: 10.1109/JSYST.2022.3215279.
[13]	TYROVOLAS D, MEKIKIS P V, TEGOS S A, et al. Energy-aware design of UAV-mounted RIS networks for IoT data collection[J]. IEEE Transactions on Communications, 2023, 71(2): 1168–1178. doi: 10.1109/TCOMM.2022.3229672.
[14]	LIU Jianghu and ZHANG Hongtao. Height-fixed UAV enabled energy-efficient data collection in RIS-aided wireless sensor networks[J]. IEEE Transactions on Wireless Communications, 2023, 22(11): 7452–7463. doi: 10.1109/TWC.2023.3250988.
[15]	ALMASOUD A M. Robust anti-jamming technique for UAV data collection in IoT using landing platforms and RIS[J]. IEEE Access, 2023, 11: 70635–70651. doi: 10.1109/ACCESS.2023.3294596.
[16]	YANG Bowen, YU Yao, LI Jianqi, et al. An AoI-guaranteed sensor data collection strategy for RIS-assisted UAV communication system[C]. Proceedings of 2023 IEEE/CIC International Conference on Communications in China, Dalian, China, 2023: 1–6. doi: 10.1109/ICCC57788.2023.10233285.
[17]	SAMIR M, ELHATTAB M, ASSI C, et al. Optimizing age of information through aerial reconfigurable intelligent surfaces: A deep reinforcement learning approach[J]. IEEE Transactions on Vehicular Technology, 2021, 70(4): 3978–3983. doi: 10.1109/TVT.2021.3063953.
[18]	HUANG Hongli, LIU Juan, and XIE Lingfu. Intelligent reflecting surface-assisted fresh data collection in UAV communications[C]. Proceedings of the 11th International Conference in Communications, Signal Processing, and Systems, Singapore, Singapore, 2022: 189–197. doi: 10.1007/978-981-99-2362-5_24.
[19]	XIAO Xiongbing, WANG Xiumin, and LIN Weiwei. Joint AoI-aware UAVs trajectory planning and data collection in UAV-based IoT systems: A deep reinforcement learning approach[J]. IEEE Transactions on Consumer Electronics, 2024, 70(4): 6484–6495. doi: 10.1109/TCE.2024.3440406.
[20]	JIANG Wenwen, AI Bo, LI Mushu, et al. Average age-of-information minimization in aerial IRS-assisted data delivery[J]. IEEE Internet of Things Journal, 2023, 10(17): 15133–15146. doi: 10.1109/JIOT.2023.3264618.
[21]	CHEN Zhen, GUO Yeyong, ZHANG Peichang, et al. Physical layer security improvement for hybrid RIS-assisted MIMO communications[J]. IEEE Communications Letters, 2024, 28(11): 2493–2497. doi: 10.1109/LCOMM.2024.3427010.
[22]	RUAN Chengyao, ZHANG Zaichen, JIANG Hao, et al. Wideband near-field channel covariance estimation for XL-MIMO systems in the face of beam split[J]. IEEE Transactions on Vehicular Technology. doi: 10.1109/TVT.2024.3471733.
[23]	QIU Chen, WEI Zhiqing, YUAN Xin, et al. Multiple UAV-mounted base station placement and user association with joint fronthaul and backhaul optimization[J]. IEEE Transactions on Communications, 2020, 68(9): 5864–5877. doi: 10.1109/TCOMM.2020.3001136.
[24]	SHI Wangqi, JIANG Hao, XIONG Baiping, et al. RIS-empowered V2V communications: Three-dimensional beam domain channel modeling and analysis[J]. IEEE Transactions on Wireless Communications, 2024, 23(11): 15844–15857. doi: 10.1109/TWC.2024.3434568.
[25]	WEI Zhiqiang, CAI Yuanxin, SUN Zhuo, et al. Sum-rate maximization for IRS-assisted UAV OFDMA communication systems[J]. IEEE Transactions on Wireless Communications, 2021, 20(4): 2530–2550. doi: 10.1109/TWC.2020.3042977.
[26]	XU Peng, NIU Wenqi, CHEN Gaojie, et al. Performance analysis of RIS-assisted systems with statistical channel state information[J]. IEEE Transactions on Vehicular Technology, 2022, 71(1): 1089–1094. doi: 10.1109/TVT.2021.3126374.
[27]	JIANG Hao, SHI Wangqi, ZHANG Zaichen, et al. Large-scale RIS enabled air-ground channels: Near-field modeling and analysis[J]. arXiv: 2403.12781, 2024. (查阅网上资料, 未能确认文献类型, 请确认) .
[28]	LIN Na, TANG Hailun, ZHAO Liang, et al. A PDDQNLP algorithm for energy efficient computation offloading in UAV-assisted MEC[J]. IEEE Transactions on Wireless Communications, 2023, 22(12): 8876–8890. doi: 10.1109/TWC.2023.3266497.
[29]	CHEN Guqiao, CHENG Changjun, XU Xiaoli, et al. Minimizing the age of information for data collection by cellular-connected UAV[J]. IEEE Transactions on Vehicular Technology, 2023, 72(7): 9631–9635. doi: 10.1109/TVT.2023.3249747.
[30]	WANG Xu, WANG Sen, LIANG Xingxing, et al. Deep reinforcement learning: A survey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(4): 5064–5078. doi: 10.1109/TNNLS.2022.3207346.
[31]	MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529–533. doi: 10.1038/nature14236.
[32]	VAN HASSELT H, GUEZ A, and SILVER D. Deep reinforcement learning with double q-learning[C]. Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, USA, 2016: 2094–2100. doi: 10.1609/aaai.v30i1.10295.
[33]	WANG Liang, WANG Kezhi, PAN Cunhua, et al. Joint trajectory and passive beamforming design for intelligent reflecting surface-aided UAV communications: A deep reinforcement learning approach[J]. IEEE Transactions on Mobile Computing, 2023, 22(11): 6543–6553. doi: 10.1109/TMC.2022.3200998.
[34]	MEI Haibo, YANG Kun, LIU Qiang, et al. 3D-trajectory and phase-shift design for RIS-assisted UAV systems using deep reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2022, 71(3): 3020–3029. doi: 10.1109/TVT.2022.3143839.