Real-time Task Scheduling for Multi-access Edge Computing-enabled AI Quality Inspection Systems
-
摘要: AI质检是智能制造的重要环节,其设备在进行产品质量检测时会产生大量计算密集型和时延敏感型任务。由于设备计算能力不足,执行检测任务时延较大,极大影响生产效率。多接入边缘计算(MEC)通过将任务卸载至边缘服务器为设备提供就近算力,提升任务执行效率。然而,系统中存在信道变化和任 务随机到达等动态因素,极大影响卸载效率,给任务调度带来了挑战。该文面向多接入边缘计算赋能的AI质检任务调度系统,研究了联合任务调度与资源分配的长期时延最小化问题。由于该问题状态空间大、动作空间包含连续变量,该文提出运用深度确定性策略梯度(DDPG)进行实时任务调度算法设计。所设计算法可基于系统实时状态信息给出最优决策。仿真结果表明,与基准算法相比,该文所提算法具有更好的性能表现和更小的任务执行时延。Abstract: AI-based quality inspection is an important part of intelligent manufacturing, where the devices produce a large amount of computation-intensive and time-sensitive tasks. Owing to the insufficient computation capability of end devices, the latency to execute these inspection tasks is large, which greatly affects manufacturing efficiency. To this end, Multi-access Edge Computing (MEC) is proposed to provide computation resources through offloading tasks to the edge servers deployed nearby. The execution efficiency is therefore improved. However, the dynamic channel state and random task arrival greatly impact the task offloading efficiency and consequently bring challenges to task scheduling. In this paper, the joint task scheduling and resource allocation problem with the purpose of minimizing the long-term delay of MEC-enabled system is studied. As the state space of the problem is large and the action space contains continuous variables, a Deep Deterministic Policy Gradient (DDPG) based real-time task scheduling algorithm is proposed. The proposed algorithm can make optimal decision with real-time system state information. Simulation results confirm the promising performance of the proposed algorithm, which achieves lower task execution latency than that of the benchmark algorithm.
-
算法1 基于DDPG的AI质检任务实时调度算法 输入:估计网络参数$ {\theta _0} $和目标网络参数$ \theta {'_0} $; 其他基本参数$ \gamma ,N,T,{f^l},{f^c},K,\lambda ,B,{N_0},\xi ,\varepsilon $; 输出:训练完成的Actor网络模型参数 (1) For ep = 1, 2, ···, K : (2) 初始化AI质检系统环境,得到环境的初始状态 s(0); (3) For t = 1, 2, ···, T : (4) 根据Actor网络的输出叠加 OU 噪声后选择一个动
作a(t) 输入环境;(5) 观测AI质检系统环境的输出奖励和下一时刻的状态; (6) 将元组$\left( {{\boldsymbol{s}}(t),{\boldsymbol{a}}(t),r(t),{\boldsymbol{s}}(t + 1)} \right)$存储到经验池中; (7) 从经验池中选择小批量数据; (8) 按式 (13) 更新估计网络的Actor网络参数; (9) 按式 (14) 更新估计网络的Critic网络参数; (10) 按式 (15) 更新目标网络的参数; (11) End (12) End 表 1 仿真参数设置
参数名 参数值 参数名 参数值 AI质检设备数量 N [6, 14] MEC服务器CPU频率 f c [4, 8] GHz 任务数据量大小 di [30, 50] KB 设备的CPU频率 f l 0.5 GHz 任务所需计算资源大小 ki [1 800, 2 600] cycle/Byte 设备的传输功率 pn 25 dBm 子信道带宽 B 1 MHz 高斯噪声功率谱密度 N0 –174 dBm/Hz 时隙长度 Ts 100 ms 经验池大小 200 000 回合周期数 T 1 000 溢出惩罚参数 ξ 10 折扣因子$\gamma $ 0.99 路径损耗常数${\beta _1}$ 10–12.7 任务队列最大长度 10 路径损耗指数${\beta _2}$ 3 设备任务平均到达率 λ [4, 10] s–1 设备到基站距离d [0,500] m 表 2 算法参数设置
网络 参数名 参数值 Actor 学习率 [10–5, 10–4] 隐藏层个数 3 隐藏层神经元数量 64, 64, 64 激活函数 LeakyReLU, Softmax, Sigmoid 软替换策略参数 ε 0.01 Critic 学习率 [10–5, 10–4] 隐藏层个数 3 隐藏层神经元数量 64, 64, 64 激活函数 LeakyReLU, Tanh 软替换策略参数 ε 0.01 -
[1] 周华, 郑荣, 肖荣. 工业场景下AI质检关键技术及平台架构研究[J]. 现代信息科技, 2022, 6(5): 149–151,156. doi: 10.19850/j.cnki.2096-4706.2022.05.039.ZHOU Hua, ZHENG Rong, and XIAO Rong. Research on key technology and platform architecture of AI quality inspection under industrial scene[J]. Modern Information Technology, 2022, 6(5): 149–151,156. doi: 10.19850/j.cnki.2096-4706.2022.05.039. [2] 蒋音. 深度学习技术开启工业AI质检新范式[J]. 大数据时代, 2022(11): 38–48.JIANG Yin. Deep learning offers a new paradigm of quality inspection supported by industrial AI[J]. Big Data Time, 2022(11): 38–48. [3] DAI Yueyue, ZHANG Ke, MAHARJAN S, et al. Deep reinforcement learning for stochastic computation offloading in digital twin networks[J]. IEEE Transactions on Industrial Informatics, 2021, 17(7): 4968–4977. doi: 10.1109/TII.2020.3016320. [4] 胡致远, 胡文前, 李香, 等. 面向业务可达性的广域工业互联网调度算法研究[J]. 电子与信息学报, 2021, 43(9): 2608–2616. doi: 10.11999/JEIT200583.HU Zhiyuan, HU Wenqian, LI Xiang, et al. Research on wide area industrial internet scheduling algorithm based on service reachability[J]. Journal of Electronics &Information Technology, 2021, 43(9): 2608–2616. doi: 10.11999/JEIT200583. [5] BAHRAMI M. Cloud computing for emerging mobile cloud apps[C]. 2015 3rd IEEE International Conference on Mobile Cloud Computing, Services, and Engineering, San Francisco, USA, 2015: 4–5. [6] ZHANG Fan, HAN Guanjie, LIU Li, et al. Deep reinforcement learning based cooperative partial task offloading and resource allocation for IIoT applications[J]. IEEE Transactions on Network Science and Engineering, 2022: 1. [7] MAO Yuyi, YOU Changsheng, ZHANG Jun, et al. A survey on mobile edge computing: The communication perspective[J]. IEEE Communications Surveys & Tutorials, 2017, 19(4): 2322–2358. doi: 10.1109/COMST.2017.2745201. [8] 李一倩, 刘留, 李慧婷, 等. 工业物联网无线信道特性研究[J]. 物联网学报, 2019, 3(4): 34–47. doi: 10.11959/j.issn.2096-3750.2019.00130.LI Yiqian, LIU Liu, LI Huiting, et al. Research on characteristics of industrial IoT wireless channel[J]. Chinese Journal on Internet of Things, 2019, 3(4): 34–47. doi: 10.11959/j.issn.2096-3750.2019.00130. [9] 张克, 刘留, 袁泽, 等. 工业物联网无线信道与噪声特性[J]. 电信科学, 2018, 34(8): 87–97. doi: 10.11959/j.issn.1000-0801.2018217.ZHANG Ke, LIU Liu, YUAN Ze, et al. Wireless channel and noise characteristics in industrial internet of things[J]. Telecommunications Science, 2018, 34(8): 87–97. doi: 10.11959/j.issn.1000-0801.2018217. [10] GUO Kai, YANG Mingcong, ZHANG Yongbing, et al. Joint computation offloading and bandwidth assignment in cloud-assisted edge computing[J]. IEEE Transactions on Cloud Computing, 2022, 10(1): 451–460. doi: 10.1109/TCC.2019.2950395. [11] YANG Lei, LIU Bo, CAO Jiannong, et al. Joint computation partitioning and resource allocation for latency sensitive applications in mobile edge clouds[J]. IEEE Transactions on Services Computing, 2021, 14(5): 1439–1452. doi: 10.1109/TSC.2018.2890603. [12] 刘斐, 曹钰杰, 章国安. 车联网场景下移动边缘计算协作式资源分配策略[J]. 电讯技术, 2021, 61(7): 858–864. doi: 10.3969/j.issn.1001-893x.2021.07.012.LIU Fei, CAO Yujie, and ZHANG Guoan. Collaborative resource allocation strategy for mobile edge computing in vehicular networks[J]. Telecommunication Engineering, 2021, 61(7): 858–864. doi: 10.3969/j.issn.1001-893x.2021.07.012. [13] 周天清, 曾新亮, 胡海琴. 基于混合粒子群算法的计算卸载成本优化[J]. 电子与信息学报, 2022, 44(9): 3065–3074. doi: 10.11999/JEIT211390.ZHOU Tianqing, ZENG Xinliang, and HU Haiqin. Computation offloading cost optimization based on hybrid particle swarm optimization algorithm[J]. Journal of Electronics &Information Technology, 2022, 44(9): 3065–3074. doi: 10.11999/JEIT211390. [14] 周天清, 胡海琴, 曾新亮. NOMA-MEC系统中基于改进遗传算法的协作式计算卸载与资源管理[J]. 电子与信息学报, 2022, 44(9): 3014–3023. doi: 10.11999/JEIT220306.ZHOU Tianqing, HU Haiqin, and ZENG Xinliang. Cooperative computation offloading and resource management based on improved genetic algorithm in NOMA-MEC systems[J]. Journal of Electronics &Information Technology, 2022, 44(9): 3014–3023. doi: 10.11999/JEIT220306. [15] LUO Quyuan, LI Changle, LUAN T H, et al. Collaborative data scheduling for vehicular edge computing via deep reinforcement learning[J]. IEEE Internet of Things Journal, 2020, 7(10): 9637–9650. doi: 10.1109/JIOT.2020.2983660. [16] ZHANG Weiting, YANG Dong, PENG Haixia, et al. Deep reinforcement learning based resource management for DNN inference in industrial IoT[J]. IEEE Transactions on Vehicular Technology, 2021, 70(8): 7605–7618. doi: 10.1109/TVT.2021.3068255. [17] CHEN Ying, LIU Zhiyong, ZHANG Yongchao, et al. Deep reinforcement learning-based dynamic resource management for mobile edge computing in industrial internet of things[J]. IEEE Transactions on Industrial Informatics, 2021, 17(7): 4925–4934. doi: 10.1109/TII.2020.3028963. [18] YU Shuai, CHEN Xu, ZHOU Zhi, et al. When deep reinforcement learning meets federated learning: Intelligent multitimescale resource management for multiaccess edge computing in 5G ultradense network[J]. IEEE Internet of Things Journal, 2021, 8(4): 2238–2251. doi: 10.1109/JIOT.2020.3026589. [19] SUTTON R S and BARTO A G. Reinforcement Learning: An Introduction[M]. 2nd ed. Cambridge: Bradford Books, 2018: 62–64. [20] LIU Binghong, LIU Chenxi and PENG Mugen. Computation offloading and resource allocation in unmanned aerial vehicle networks[J]. IEEE Transactions on Vehicular Technology, 2023, 72(4): 4981–4995. doi: 10.1109/TVT.2022.3222907. [21] DAI Bin, NIU Jianwei, REN Tao, et al. Toward mobility-aware computation offloading and resource allocation in end–edge–cloud orchestrated computing[J]. IEEE Internet of Things Journal, 2022, 9(19): 19450–19462. doi: 10.1109/JIOT.2022.3168036. [22] QIAO Guanhua, LENG Supeng, MAHARJAN S, et al. Deep reinforcement learning for cooperative content caching in vehicular edge computing and networks[J]. IEEE Internet of Things Journal, 2020, 7(1): 247–257. doi: 10.1109/JIOT.2019.2945640.