多接入边缘计算赋能的AI质检系统任务实时调度策略

周晓天; 孙上; 张海霞; 邓伊琴; 鲁彬彬

doi:10.11999/JEIT230129

多接入边缘计算赋能的AI质检系统任务实时调度策略

doi: 10.11999/JEIT230129

1.
山东大学控制科学与工程学院济南 250061
2.
山东省无线通信技术重点实验室济南 250100

基金项目: 国家自然科学基金(61860206005, U22A2003, 61971270)

详细信息

作者简介:
周晓天：男，副教授，研究方向为无线通信与网络、边缘计算与智能通信等

孙上：女，硕士生，研究方向为物联网、边缘计算等

张海霞：女，教授，研究方向为无线通信与网络、无线资源管理、智能通信技术等

邓伊琴：女，博士后，研究方向为边缘计算、车联网、无线资源优化等

鲁彬彬：男，硕士，研究方向为车联网、计算卸载、资源优化等

通讯作者:
张海霞　haixia.zhang@sdu.edu.cn

中图分类号: TN929.5; TP18
计量
- 文章访问数: 611
- HTML全文浏览量: 369
- PDF下载量: 93
- 被引次数: 0
出版历程
- 收稿日期: 2023-03-03
- 修回日期: 2023-08-15
- 网络出版日期: 2023-08-17
- 刊出日期: 2024-02-29

Real-time Task Scheduling for Multi-access Edge Computing-enabled AI Quality Inspection Systems

1.
School of Control Science and Engineering, Shandong University, Jinan 250061, China
2.
Shandong Key Laboratory of Wireless Communication Technologies, Shandong University, Jinan 250100, China

Funds: The National Natural Science Foundation of China (61860206005, U22A2003, 61971270)

摘要

摘要: AI质检是智能制造的重要环节，其设备在进行产品质量检测时会产生大量计算密集型和时延敏感型任务。由于设备计算能力不足，执行检测任务时延较大，极大影响生产效率。多接入边缘计算(MEC)通过将任务卸载至边缘服务器为设备提供就近算力，提升任务执行效率。然而，系统中存在信道变化和任务随机到达等动态因素，极大影响卸载效率，给任务调度带来了挑战。该文面向多接入边缘计算赋能的AI质检任务调度系统，研究了联合任务调度与资源分配的长期时延最小化问题。由于该问题状态空间大、动作空间包含连续变量，该文提出运用深度确定性策略梯度(DDPG)进行实时任务调度算法设计。所设计算法可基于系统实时状态信息给出最优决策。仿真结果表明，与基准算法相比，该文所提算法具有更好的性能表现和更小的任务执行时延。
- 多接入边缘计算 /
- 任务调度 /
- 资源分配 /
- 深度强化学习 /
- AI质检系统
Abstract: AI-based quality inspection is an important part of intelligent manufacturing, where the devices produce a large amount of computation-intensive and time-sensitive tasks. Owing to the insufficient computation capability of end devices, the latency to execute these inspection tasks is large, which greatly affects manufacturing efficiency. To this end, Multi-access Edge Computing (MEC) is proposed to provide computation resources through offloading tasks to the edge servers deployed nearby. The execution efficiency is therefore improved. However, the dynamic channel state and random task arrival greatly impact the task offloading efficiency and consequently bring challenges to task scheduling. In this paper, the joint task scheduling and resource allocation problem with the purpose of minimizing the long-term delay of MEC-enabled system is studied. As the state space of the problem is large and the action space contains continuous variables, a Deep Deterministic Policy Gradient (DDPG) based real-time task scheduling algorithm is proposed. The proposed algorithm can make optimal decision with real-time system state information. Simulation results confirm the promising performance of the proposed algorithm, which achieves lower task execution latency than that of the benchmark algorithm.
- Multi-access Edge Computing (MEC) /
- Task scheduling /
- Resource allocation /
- Deep reinforcement learning /
- AI-based quality inspection system

HTML全文

图 1 MEC赋能的AI质检系统任务调度结构图

下载: 全尺寸图片幻灯片

图 2 算法结构图

下载: 全尺寸图片幻灯片

图 3 不同学习率下的累积奖励

下载: 全尺寸图片幻灯片

图 4 系统长期任务执行时延方案对比

下载: 全尺寸图片幻灯片

图 5 系统平均任务执行时延方案对比

下载: 全尺寸图片幻灯片

算法1 基于DDPG的AI质检任务实时调度算法
输入：估计网络参数$ {\theta _0} $和目标网络参数$ \theta {'_0} $;
其他基本参数$ \gamma ,N,T,{f^l},{f^c},K,\lambda ,B,{N_0},\xi ,\varepsilon $;
输出：训练完成的Actor网络模型参数
(1) For ep = 1, 2, ···, K :
(2) 　　初始化AI质检系统环境，得到环境的初始状态 s(0);
(3) 　　For t = 1, 2, ···, T :
(4) 　　　　根据Actor网络的输出叠加 OU 噪声后选择一个动　　　　　　作a(t) 输入环境;
(5) 　　　　观测AI质检系统环境的输出奖励和下一时刻的状态;
(6) 　　　　将元组$\left( {{\boldsymbol{s}}(t),{\boldsymbol{a}}(t),r(t),{\boldsymbol{s}}(t + 1)} \right)$存储到经验池中;
(7) 　　　　从经验池中选择小批量数据;
(8) 　　　　按式 (13) 更新估计网络的Actor网络参数;
(9) 　　　　按式 (14) 更新估计网络的Critic网络参数;
(10) 　　　按式 (15) 更新目标网络的参数;
(11) 　 End
(12) End

下载: 导出CSV

表 1 仿真参数设置

参数名	参数值	参数名	参数值
AI质检设备数量 N	[6, 14]	MEC服务器CPU频率 f^c	[4, 8] GHz
任务数据量大小 d_i	[30, 50] KB	设备的CPU频率 f^l	0.5 GHz
任务所需计算资源大小 k_i	[1 800, 2 600] cycle/Byte	设备的传输功率 p_n	25 dBm
子信道带宽 B	1 MHz	高斯噪声功率谱密度 N₀	–174 dBm/Hz
时隙长度 T_s	100 ms	经验池大小	200 000
回合周期数 T	1 000	溢出惩罚参数 ξ	10
折扣因子$\gamma $	0.99	路径损耗常数${\beta _1}$	10^–12.7
任务队列最大长度	10	路径损耗指数${\beta _2}$	3
设备任务平均到达率 λ	[4, 10] s^–1	设备到基站距离d	[0,500] m

下载: 导出CSV

表 2 算法参数设置

网络	参数名	参数值
Actor	学习率	[10^–5, 10^–4]
	隐藏层个数	3
	隐藏层神经元数量	64, 64, 64
	激活函数	LeakyReLU, Softmax, Sigmoid
	软替换策略参数 ε	0.01
Critic	学习率	[10^–5, 10^–4]
	隐藏层个数	3
	隐藏层神经元数量	64, 64, 64
	激活函数	LeakyReLU, Tanh
	软替换策略参数 ε	0.01

下载: 导出CSV

参考文献(22)

[1]	周华, 郑荣, 肖荣. 工业场景下AI质检关键技术及平台架构研究[J]. 现代信息科技, 2022, 6(5): 149–151,156. doi: 10.19850/j.cnki.2096-4706.2022.05.039. ZHOU Hua, ZHENG Rong, and XIAO Rong. Research on key technology and platform architecture of AI quality inspection under industrial scene[J]. Modern Information Technology, 2022, 6(5): 149–151,156. doi: 10.19850/j.cnki.2096-4706.2022.05.039.
[2]	蒋音. 深度学习技术开启工业AI质检新范式[J]. 大数据时代, 2022(11): 38–48. JIANG Yin. Deep learning offers a new paradigm of quality inspection supported by industrial AI[J]. Big Data Time, 2022(11): 38–48.
[3]	DAI Yueyue, ZHANG Ke, MAHARJAN S, et al. Deep reinforcement learning for stochastic computation offloading in digital twin networks[J]. IEEE Transactions on Industrial Informatics, 2021, 17(7): 4968–4977. doi: 10.1109/TII.2020.3016320.
[4]	胡致远, 胡文前, 李香, 等. 面向业务可达性的广域工业互联网调度算法研究[J]. 电子与信息学报, 2021, 43(9): 2608–2616. doi: 10.11999/JEIT200583. HU Zhiyuan, HU Wenqian, LI Xiang, et al. Research on wide area industrial internet scheduling algorithm based on service reachability[J]. Journal of Electronics &Information Technology, 2021, 43(9): 2608–2616. doi: 10.11999/JEIT200583.
[5]	BAHRAMI M. Cloud computing for emerging mobile cloud apps[C]. 2015 3rd IEEE International Conference on Mobile Cloud Computing, Services, and Engineering, San Francisco, USA, 2015: 4–5.
[6]	ZHANG Fan, HAN Guanjie, LIU Li, et al. Deep reinforcement learning based cooperative partial task offloading and resource allocation for IIoT applications[J]. IEEE Transactions on Network Science and Engineering, 2022: 1.
[7]	MAO Yuyi, YOU Changsheng, ZHANG Jun, et al. A survey on mobile edge computing: The communication perspective[J]. IEEE Communications Surveys & Tutorials, 2017, 19(4): 2322–2358. doi: 10.1109/COMST.2017.2745201.
[8]	李一倩, 刘留, 李慧婷, 等. 工业物联网无线信道特性研究[J]. 物联网学报, 2019, 3(4): 34–47. doi: 10.11959/j.issn.2096-3750.2019.00130. LI Yiqian, LIU Liu, LI Huiting, et al. Research on characteristics of industrial IoT wireless channel[J]. Chinese Journal on Internet of Things, 2019, 3(4): 34–47. doi: 10.11959/j.issn.2096-3750.2019.00130.
[9]	张克, 刘留, 袁泽, 等. 工业物联网无线信道与噪声特性[J]. 电信科学, 2018, 34(8): 87–97. doi: 10.11959/j.issn.1000-0801.2018217. ZHANG Ke, LIU Liu, YUAN Ze, et al. Wireless channel and noise characteristics in industrial internet of things[J]. Telecommunications Science, 2018, 34(8): 87–97. doi: 10.11959/j.issn.1000-0801.2018217.
[10]	GUO Kai, YANG Mingcong, ZHANG Yongbing, et al. Joint computation offloading and bandwidth assignment in cloud-assisted edge computing[J]. IEEE Transactions on Cloud Computing, 2022, 10(1): 451–460. doi: 10.1109/TCC.2019.2950395.
[11]	YANG Lei, LIU Bo, CAO Jiannong, et al. Joint computation partitioning and resource allocation for latency sensitive applications in mobile edge clouds[J]. IEEE Transactions on Services Computing, 2021, 14(5): 1439–1452. doi: 10.1109/TSC.2018.2890603.
[12]	刘斐, 曹钰杰, 章国安. 车联网场景下移动边缘计算协作式资源分配策略[J]. 电讯技术, 2021, 61(7): 858–864. doi: 10.3969/j.issn.1001-893x.2021.07.012. LIU Fei, CAO Yujie, and ZHANG Guoan. Collaborative resource allocation strategy for mobile edge computing in vehicular networks[J]. Telecommunication Engineering, 2021, 61(7): 858–864. doi: 10.3969/j.issn.1001-893x.2021.07.012.
[13]	周天清, 曾新亮, 胡海琴. 基于混合粒子群算法的计算卸载成本优化[J]. 电子与信息学报, 2022, 44(9): 3065–3074. doi: 10.11999/JEIT211390. ZHOU Tianqing, ZENG Xinliang, and HU Haiqin. Computation offloading cost optimization based on hybrid particle swarm optimization algorithm[J]. Journal of Electronics &Information Technology, 2022, 44(9): 3065–3074. doi: 10.11999/JEIT211390.
[14]	周天清, 胡海琴, 曾新亮. NOMA-MEC系统中基于改进遗传算法的协作式计算卸载与资源管理[J]. 电子与信息学报, 2022, 44(9): 3014–3023. doi: 10.11999/JEIT220306. ZHOU Tianqing, HU Haiqin, and ZENG Xinliang. Cooperative computation offloading and resource management based on improved genetic algorithm in NOMA-MEC systems[J]. Journal of Electronics &Information Technology, 2022, 44(9): 3014–3023. doi: 10.11999/JEIT220306.
[15]	LUO Quyuan, LI Changle, LUAN T H, et al. Collaborative data scheduling for vehicular edge computing via deep reinforcement learning[J]. IEEE Internet of Things Journal, 2020, 7(10): 9637–9650. doi: 10.1109/JIOT.2020.2983660.
[16]	ZHANG Weiting, YANG Dong, PENG Haixia, et al. Deep reinforcement learning based resource management for DNN inference in industrial IoT[J]. IEEE Transactions on Vehicular Technology, 2021, 70(8): 7605–7618. doi: 10.1109/TVT.2021.3068255.
[17]	CHEN Ying, LIU Zhiyong, ZHANG Yongchao, et al. Deep reinforcement learning-based dynamic resource management for mobile edge computing in industrial internet of things[J]. IEEE Transactions on Industrial Informatics, 2021, 17(7): 4925–4934. doi: 10.1109/TII.2020.3028963.
[18]	YU Shuai, CHEN Xu, ZHOU Zhi, et al. When deep reinforcement learning meets federated learning: Intelligent multitimescale resource management for multiaccess edge computing in 5G ultradense network[J]. IEEE Internet of Things Journal, 2021, 8(4): 2238–2251. doi: 10.1109/JIOT.2020.3026589.
[19]	SUTTON R S and BARTO A G. Reinforcement Learning: An Introduction[M]. 2nd ed. Cambridge: Bradford Books, 2018: 62–64.
[20]	LIU Binghong, LIU Chenxi and PENG Mugen. Computation offloading and resource allocation in unmanned aerial vehicle networks[J]. IEEE Transactions on Vehicular Technology, 2023, 72(4): 4981–4995. doi: 10.1109/TVT.2022.3222907.
[21]	DAI Bin, NIU Jianwei, REN Tao, et al. Toward mobility-aware computation offloading and resource allocation in end–edge–cloud orchestrated computing[J]. IEEE Internet of Things Journal, 2022, 9(19): 19450–19462. doi: 10.1109/JIOT.2022.3168036.
[22]	QIAO Guanhua, LENG Supeng, MAHARJAN S, et al. Deep reinforcement learning for cooperative content caching in vehicular edge computing and networks[J]. IEEE Internet of Things Journal, 2020, 7(1): 247–257. doi: 10.1109/JIOT.2019.2945640.