Online Trajectory Optimization for the UAV-Mounted Base Stations
-
摘要: 针对离线的无人机(UAV)基站飞行路线设计无法满足随机的、动态的地面用户通信请求难题，该文研究了飞行路线在线优化设计算法。考虑单个无人机空中基站为两个地面用户提供无线通信服务，通过在线实时优化无人机的飞行路线实现最小化与地面用户的平均通信时延。首先，由于系统的无人机的状态和动作是连续的，将问题转化成一个马尔可夫决策过程(MDP)；然后，把单次通信时延引入到动作价值函数中；最后分别采用强化学习中蒙特卡罗和Q-Learning算法来实现无人机的飞行路线在线优化。仿真结果表明，所提出的在线优化的平均时延性能优于“固定位置”和“贪婪算法”的时延计算结果。Abstract: Considering dealing with the problem of random and dynamic communication requests of ground users in a UAV(Unmanned Aerial Vehicle) mounted base station communication system, which can not be tackled by an offline trajectory design scheme, an online trajectory optimization algorithm is proposed for the UAV-mounted base station. In the considered system, a single UAV is utilized as an aerial base station to provide wireless communication service to two ground users. The problem of minimizing the average communication delay of the ground users via optimizing the UAV’s trajectory is considered. First, it is shown that the problem can be casted as a Markov Decision Process (MDP), and then the delay of one single communication is introduced into the action value function. Finally, the Monte Carlo and Q-Learning algorithms from the reinforcement learning technology are respectively adopted to realize the online trajectory optimization. Simulation results show that the proposed algorithm outperforms the “fixed position” and “greedy algorithm” schemes.
-
[1] ZENG Yong, ZHANG Rui, and LIM T J. Wireless communications with unmanned aerial vehicles: Opportunities and challenges[J]. IEEE Communications Magazine, 2016, 54(5): 36–42. doi: 10.1109/MCOM.2016.7470933 [2] Paving the path to 5G: Optimizing commercial LTE networks for drone communication[EB/OL]. https://www.qualcomm.com/news/onq/2016/09/06/paving-path-5g-optimizing-commercial-lte-networks-drone-communication, 2016. [3] ZHANG Guangchi, WU Qingqing, CUI Miao, et al. Securing UAV communications via joint trajectory and power control[J]. IEEE Transactions on Wireless Communications, 2019, 18(2): 1376–1389. doi: 10.1109/TWC.2019.2892461 [4] WU Qingqing, ZENG Yong, and ZHANG Rui. Joint trajectory and communication design for multi-UAV enabled wireless networks[J]. IEEE Transactions on Wireless Communications, 2018, 17(3): 2109–2121. doi: 10.1109/TWC.2017.2789293 [5] ZENG Yong, ZHANG Rui, and LIM T J. Throughput maximization for UAV-enabled mobile relaying systems[J]. IEEE Transactions on Communications, 2016, 64(12): 4983–4996. doi: 10.1109/TCOMM.2016.2611512 [6] ZENG Yong, LYU Jiangbin, and ZHANG Rui. Cellular-connected UAV: Potential, challenges, and promising technologies[J]. IEEE Wireless Communications, 2019, 26(1): 120–127. doi: 10.1109/MWC.2018.1800023 [7] LYU Jiangbin, ZENG Yong, ZHANG Rui, et al. Placement optimization of UAV-mounted mobile base stations[J]. IEEE Communications Letters, 2017, 21(3): 604–607. doi: 10.1109/LCOMM.2016.2633248 [8] ZHAN Cheng, ZENG Yong, and ZHANG Rui. Energy-efficient data collection in UAV enabled wireless sensor network[J]. IEEE Wireless Communications Letters, 2018, 7(3): 328–331. doi: 10.1109/LWC.2017.2776922 [9] ZHANG Guangchi, YAN Haiqiang, ZENG Yong, et al. Trajectory optimization and power allocation for multi-hop UAV relaying communications[J]. IEEE Access, 2018, 6: 48566–48576. doi: 10.1109/ACCESS.2018.2868117 [10] ZENG Yong and XU Xiaoli. Path design for cellular-connected UAV with reinforcement learning[EB/OL]. http://arxiv.org/abs/1905.03440, 2019. [11] 黄长强, 赵克新, 韩邦杰, 等. 一种近似动态规划的无人机机动决策方法[J]. 电子与信息学报, 2018, 40(10): 2447–2452. doi: 10.11999/JEIT180068HUANG Changqiang, ZHAO Kexin, HAN Bangjie, et al. Maneuvering decision-making method of UAV based on approximate dynamic programming[J]. Journal of Electronics &Information Technology, 2018, 40(10): 2447–2452. doi: 10.11999/JEIT180068 [12] BLISS M and MICHELUSI N. Trajectory optimization for rotary-wing UAVs in wireless networks with random requests[EB/OL]. http://arxiv.org/abs/1905.01755, 2019. [13] SUTTON R S and BARTO A G. Reinforcement Learning: An Introduction[M]. 2nd ed. Cambridge: MIT Press, 2018: 1–130. [14] LIU Xiao, LIU Yuanwei, and CHEN Yue. Reinforcement learning in multiple-UAV networks: Deployment and movement design[J]. IEEE Transactions on Vehicular Technology, 2019, 68(8): 8036–8049. doi: 10.1109/TVT.2019.2922849 [15] KHAMIDEHI B and SOUSA E S. Reinforcement learning-based trajectory design for the aerial base stations[EB/OL]. https://arxiv.org/abs/1906.09550, 2019. [16] ZENG Yong and ZHANG Rui. Energy-efficient UAV communication with trajectory optimization[J]. IEEE Transactions on Wireless Communications, 2017, 16(6): 3747–3760. doi: 10.1109/TWC.2017.2688328 -
下载:
图(7)
计量
- 文章访问数: 827
- HTML全文浏览量: 561
- PDF下载量: 148
- 被引次数: 0
下载:
