Energy Optimization for Computing Reuse in Unmanned Aerial Vehicle-assisted Edge Computing Systems
-
摘要: 针对复杂地形下时延敏感任务对终端用户的计算需求激增问题，该文提出一种无人机(UAV)辅助的移动边缘计算可重用任务的协同计算卸载方案。首先，通过联合优化用户卸载策略、用户传输功率、无人机上服务器分配、用户设备的计算频率和无人机服务器的计算频率以及无人机的飞行轨迹，构建满足时延约束下最小化系统平均总能耗的系统模型。其次，通过深度强化学习求解该优化问题，并提出了基于柔性动作-评价(SAC)的优化算法。该算法采用最大熵的策略来鼓励探索，以增强算法的探索能力并加快训练的收敛速度。仿真结果表明，基于SAC的算法能有效降低系统的平均总能耗，并具有较好的收敛性。Abstract: To address the high computational performance demands of delay-sensitive tasks in complex terrains, the collaborative computation offloading scheme for reusable tasks in mobile edge computing with the assistance of Unmanned Aerial Vehicle (UAV) is proposed. Firstly, the minimization of the average total energy consumption is formulated by jointly optimizing user offloading, user transmission power, server assignment on UAV, computation frequencies of users and UAV servers, as well as UAV flight trajectory, while meeting the latency constraints. Secondly, a deep reinforcement learning approach is employed to solve the optimization problem, and a Soft Actor-Critic (SAC) based optimization algorithm is introduced. The SAC algorithm utilizes a maximum entropy policy to encourage exploration that enhances the algorithm’s exploration capabilities and accelerates the training convergence speed. Simulation results demonstrate that the proposed SAC algorithm effectively reduces the average total energy consumption of the system while exhibiting good convergence.
-
1 基于SAC的系统平均总能耗最小化算法
输入：最大回合数E，学习率$\beta $，折扣回报$\gamma $，时隙数N。 步骤1 初始化经验数组，Actor网络，Critic网络及目标网络参
数，随机生成用户坐标以及计算任务信息；步骤2 for episode=1:E 初始化无人机初始坐标以及初始状态$ s[0] $； for slot=1:N 更新时隙t用户设备上到达任务的信息和带宽分配情况； 根据当前策略${\pi _\varphi }$和状态$ s[t] $选择动作$a[t]$； 根据奖励函数计算$r[t]$，观察下一个状态$s[t + 1]$，并将
$\left\{ {s[t],a[t],r[t],s[t + 1]} \right\}$存储到经验回放数组；从经验回放数组中随机采样一组经验样本，根据式(16)
和式(17)分别计算损失函数${L_Q}(\theta )$,$ {L_\pi }(\varphi ) $，并更新Q网
络参数$\theta $,V网络参数$ \varphi $和温和因子$ \alpha $；每隔Z步更新目标网络参数$ {\varphi ^ - } $； 步骤3 输出用户卸载策略网络参数$ \varphi $。 
下载: 导出CSV
表 1 SAC训练参数
参数 值 参数 值 隐藏层数量$ L $ 3 惩罚值$ C $ 8 折扣回报$ \gamma $ 0.99 目标网络更新频率Z 320 最大回合数E 103 温和因子$ \alpha $初始值 0.005 学习率$ \beta $ 10–4 贪婪策略比例 0.2 批次经验大小 64 经验回放数组大小 106
下载: 导出CSV
表 2 无人机飞行功率相关参数
参数 值 参数 值 UAV叶片旋转功率$ {p_{\rm{rot}}} $ 59.03 W 空气密度$ \delta $ 1.225 kg/m3 UAV悬停功率$ {p_{\rm{hov}}} $ 79.07 W 转子盘面积$ A $ 0.5030 m2 UAV叶片尖端速度$ {v_{{\mathrm{tip}}}} $ 120 m/s 机身阻力比$ \varepsilon $ 0.6 UAV悬停时平均转子速度$ {v_{\rm{hov}}} $ 3.6 m/s 转子稳定度$ \lambda $ 0.05
下载: 导出CSV
-
[1] LIU Yaqiong, PENG Mugen, SHOU Guochu, et al. Toward edge intelligence: Multiaccess edge computing for 5G and internet of things[J]. IEEE Internet of Things Journal, 2020, 7(8): 6722–6747. doi: 10.1109/JIOT.2020.3004500 [2] 贾淑霞, 郝万明, 高梓涵, 等. 任务迁移的移动边缘计算系统中资源分配和任务卸载策略[J]. 电讯技术, 2022, 62(8): 1037–1043. doi: 10.3969/j.issn.1001-893x.2022.08.003JIA Shuxia, HAO Wanming, GAO Zihan, et al. Resource allocation and task offloading strategy in mobile edge computing system for task migration[J]. Telecommunication Engineering, 2022, 62(8): 1037–1043. doi: 10.3969/j.issn.1001-893x.2022.08.003 [3] TUN Y K, DANG T N, KIM K, et al. Collaboration in the sky: A distributed framework for task offloading and resource allocation in multi-access edge computing[J]. IEEE Internet of Things Journal, 2022, 9(23): 24221–24235. doi: 10.1109/JIOT.2022.3189000 [4] SUN Geng, LI Jiahui, LIU Yanheng, et al. Time and energy minimization communications based on collaborative beamforming for UAV networks: A multi-objective optimization method[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(11): 3555–3572. doi: 10.1109/JSAC.2021.3088720 [5] HSU Y H and GAU R H. Reinforcement learning-based collision avoidance and optimal trajectory planning in UAV communication networks[J]. IEEE Transactions on Mobile Computing, 2022, 21(1): 306–320. doi: 10.1109/TMC.2020.3003639 [6] WANG Longjie, ZHOU Qiang, and SHEN Yan. Computation efficiency maximization for UAV-assisted relaying and MEC networks in urban environment[J]. IEEE Transactions on Green Communications and Networking, 2023, 7(2): 565–578. doi: 10.1109/TGCN.2022.3222398 [7] WANG Yong, RU Zhiyang, WANG Kezhi, et al. Joint deployment and task scheduling optimization for large-scale mobile users in multi-UAV-enabled mobile edge computing[J]. IEEE Transactions on Cybernetics, 2020, 50(9): 3984–3997. doi: 10.1109/TCYB.2019.2935466 [8] WANG Di, TIAN Jie, ZHANG Haixia, et al. Task offloading and trajectory scheduling for UAV-enabled MEC networks: An optimal transport theory perspective[J]. IEEE Wireless Communications Letters, 2022, 11(1): 150–154. doi: 10.1109/LWC.2021.3122957 [9] 张广驰, 何梓楠, 崔苗. 基于深度强化学习的无人机辅助移动边缘计算系统能耗优化[J]. 电子与信息学报, 2023, 45(5): 1635–1643. doi: 10.11999/JEIT220352ZHANG Guangchi, HE Zinan, and CUI Miao. Energy consumption optimization of unmanned aerial vehicle assisted mobile edge computing systems based on deep reinforcement learning[J]. Journal of Electronics & Information Technology, 2023, 45(5): 1635–1643. doi: 10.11999/JEIT220352 [10] WESTERA W, PRADA R, MASCARENHAS S, et al. Artificial intelligence moving serious gaming: Presenting reusable game AI components[J]. Education and Information Technologies, 2020, 25(1): 351–380. doi: 10.1007/s10639-019-09968-2 [11] YANG Xuemei, LUO Hong, SUN Yan, et al. Energy-efficient collaborative offloading for multiplayer games with cache-aided MEC[C]. The IEEE International Conference on Communications, Dublin, Ireland, 2020: 1–7. [12] YANG Xuemei, LUO Hong, SUN Yan, et al. Coalitional game-based cooperative computation offloading in MEC for reusable tasks[J]. IEEE Internet of Things Journal, 2021, 8(16): 12968–12982. doi: 10.1109/JIOT.2021.3064186 [13] YANG Xuemei, LUO Hong, SUN Yan, et al. A novel hybrid-ARPPO algorithm for dynamic computation offloading in edge computing[J]. IEEE Internet of Things Journal, 2022, 9(23): 24065–24078. doi: 10.1109/JIOT.2022.3188928 [14] LIU Boyang, WAN Yiyao, ZHOU Fuhui, et al. Resource allocation and trajectory design for MISO UAV-assisted MEC networks[J]. IEEE Transactions on Vehicular Technology, 2022, 71(5): 4933–4948. doi: 10.1109/TVT.2022.3140833 [15] CHEN Peipei, LUO Xueshan, GUO Deke, et al. Secure task offloading for MEC-aided-UAV system[J]. IEEE Transactions on Intelligent Vehicles, 2023, 8(5): 3444–3457. doi: 10.1109/TIV.2022.3227367 -
下载:
图(6) / 表(3)
计量
- 文章访问数: 85
- HTML全文浏览量: 29
- PDF下载量: 24
- 被引次数: 0
