Energy Consumption Optimization of Unmanned Aerial Vehicle Assisted Mobile Edge Computing Systems Based on Deep Reinforcement Learning

ZHANG Guangchi; HE Zinan; CUI Miao

doi:10.11999/JEIT220352

Volume 45 Issue 5

May 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 > 45(5): 1635-1643

ZHANG Guangchi, HE Zinan, CUI Miao. Energy Consumption Optimization of Unmanned Aerial Vehicle Assisted Mobile Edge Computing Systems Based on Deep Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2023, 45(5): 1635-1643. doi: 10.11999/JEIT220352

Citation:

ZHANG Guangchi, HE Zinan, CUI Miao. Energy Consumption Optimization of Unmanned Aerial Vehicle Assisted Mobile Edge Computing Systems Based on Deep Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2023, 45(5): 1635-1643. doi: 10.11999/JEIT220352

Citation:

ZHANG Guangchi, HE Zinan, CUI Miao. Energy Consumption Optimization of Unmanned Aerial Vehicle Assisted Mobile Edge Computing Systems Based on Deep Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2023, 45(5): 1635-1643. doi: 10.11999/JEIT220352

PDF( 5232 KB)

Energy Consumption Optimization of Unmanned Aerial Vehicle Assisted Mobile Edge Computing Systems Based on Deep Reinforcement Learning

doi: 10.11999/JEIT220352

School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China

Funds: The Science and Technology Plan Project of Guangdong Province (2021A0505030015, 2020A050515010), The Special Support Plan for High-Level Talents of Guangdong Province (2019TQ05X409), The Open Research Project Programme of the State Key Laboratory of Internet of Things for Smart City (University of Macau) (SKL-IoTSC(UM)-2021-2023/ORPF/A04/2022)

Received Date: 2022-03-31
Rev Recd Date: 2022-07-05

Available Online: 2022-07-06

Publish Date: 2023-05-10

Abstract

Abstract

In recent years, the deployment of Unmanned Aerial Vehicles (UAVs) equipped with Mobile Edge Computing (MEC) servers to provide computing services for ground users has become an emerging method. Considering an UAV-assisted MEC system with multi-users, a scheme is investigated to minimize the average energy consumption for all users to complete their computation tasks via optimizing the trajectory of UAV and computation strategies of the users during the UAV’s whole flight duration. A Deep Reinforcement Learning (DRL)-based Soft Actor-Critic (SAC) algorithm is proposed to tackle the energy consumption optimization problem. With the iteration of the network training procedure, the best action is obtained according to the maximum entropy rule, which does not neglect any action with high reward value and thus can enhance the exploration and convergence performance of the proposed algorithm. Simulation results reveal that the proposed SAC algorithm can effectively decrease the average energy consumption of all users and achieves better stability and convergence performance, as compared to some existing baseline algorithms.
- Unmanned Aerial Vehicles (UAV) communication,
- Deep Reinforcement Learning (DRL),
- Mobile Edge Computing (MEC),
- Trajectory optimization,
- Scheduling policy

FullText(HTML)

References(19)

References

[1]	MAO Yuyi, YOU Changsheng, ZHANG Jun, et al. A survey on mobile edge computing: The communication perspective[J]. IEEE Communications Surveys & Tutorials, 2017, 19(4): 2322–2358. doi: 10.1109/COMST.2017.2745201
[2]	MAO Yuyi, ZHANG Jun, and LETAIEF K B. Dynamic computation offloading for mobile-edge computing with energy harvesting devices[J]. IEEE Journal on Selected Areas in Communications, 2016, 34(12): 3590–3605. doi: 10.1109/JSAC.2016.2611964
[3]	LIU Tianyu, CUI Miao, ZHANG Guangchi, et al. 3D trajectory and transmit power optimization for UAV-enabled multi-link relaying systems[J]. IEEE Transactions on Green Communications and Networking, 2021, 5(1): 392–405. doi: 10.1109/TGCN.2020.3048135
[4]	LYU Xinchen, TIAN Hui, NI Wei, et al. Energy-efficient admission of delay-sensitive tasks for mobile edge computing[J]. IEEE Transactions on Communications, 2018, 66(6): 2603–2616. doi: 10.1109/TCOMM.2018.2799937
[5]	WU Qingqing and ZHANG Rui. Common throughput maximization in UAV-enabled OFDMA systems with delay consideration[J]. IEEE Transactions on Communications, 2018, 66(12): 6614–6627. doi: 10.1109/TCOMM.2018.2865922
[6]	LI Zhiyang, CHEN Ming, PAN Cunhua, et al. Joint trajectory and communication design for secure UAV networks[J]. IEEE Communications Letters, 2019, 23(4): 636–639. doi: 10.1109/LCOMM.2019.2898404
[7]	LI Yuxi. Deep reinforcement learning: An overview[EB/OL]. https://arxiv.org/abs/1701.07274, 2021.
[8]	PENG Yingsheng, LIU Yong, and ZHANG Han. Deep reinforcement learning based path planning for UAV-assisted edge computing networks[C]. 2021 IEEE Wireless Communications and Networking Conference, Nanjing, China, 2021: 1–6.
[9]	SEID A M, BOATENG G O, ANOKYE S, et al. Collaborative computation offloading and resource allocation in multi-UAV-assisted IoT networks: A deep reinforcement learning approach[J]. IEEE Internet of Things Journal, 2021, 8(15): 12203–12218. doi: 10.1109/JIOT.2021.3063188
[10]	FUJIMOTO S and GU S S. A minimalist approach to offline reinforcement learning[C]. The 34th Annual Conference on Neural Information Processing Systems, Vancouver, Canada, 2021.
[11]	HAARNOJA T, ZHOU A, HARTIKAINEN K, et al. Soft actor-critic algorithms and applications[EB/OL]. https://arxiv.org/abs/1812.05905, 2021.
[12]	LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[C]. The 4th International Conference on Learning Representations, San Juan, Puerto Rico, 2021.
[13]	ZHANG Guangchi, YAN Haiqiang, ZENG Yong, et al. Trajectory optimization and power allocation for multi-hop UAV relaying communications[J]. IEEE Access, 2018, 6: 48566–48576. doi: 10.1109/ACCESS.2018.2868117
[14]	YU Zhe, GONG Yanmin, GONG Shimin, et al. Joint task offloading and resource allocation in UAV-enabled mobile edge computing[J]. IEEE Internet of Things Journal, 2020, 7(4): 3147–3159. doi: 10.1109/JIOT.2020.2965898
[15]	HUANG Yingqian, CUI Miao, ZHANG Guangchi, et al. Bandwidth, power and trajectory optimization for UAV base station networks with backhaul and user QoS constraints[J]. IEEE Access, 2020, 8: 67625–67634. doi: 10.1109/ACCESS.2020.2986075
[16]	YANG Zhaohui, PAN Cunhua, WANG Kezhi, et al. Energy efficient resource allocation in UAV-enabled mobile edge computing networks[J]. IEEE Transactions on Wireless Communications, 2019, 18(9): 4576–4589. doi: 10.1109/TWC.2019.2927313
[17]	ZHANG Guangchi, WU Qingqing, CUI Miao, et al. Securing UAV communications via joint trajectory and power control[J]. IEEE Transactions on Wireless Communications, 2019, 18(2): 1376–1389. doi: 10.1109/TWC.2019.2892461
[18]	WANG Xinhou, WANG Kezhi, WU Song, et al. Dynamic resource scheduling in mobile edge cloud with cloud radio access network[J]. IEEE Transactions on Parallel and Distributed Systems, 2018, 29(11): 2429–2445. doi: 10.1109/TPDS.2018.2832124
[19]	JIANG Feibo, WANG Kezhi, DONG Li, et al. Deep-learning-based joint resource scheduling algorithms for hybrid MEC networks[J]. IEEE Internet of Things Journal, 2020, 7(7): 6252–6265. doi: 10.1109/JIOT.2019.2954503