Energy Harvesting Assisted Intelligent Computation Offloading Method for the IoT in Mining
-
摘要: 针对计算、能量和内存资源受限的矿山物联网设备和大量时延敏感型计算任务需求的智慧矿山场景，该文提出一种能量收集(EH)辅助的矿山物联网智能计算卸载方法。通过采用移动边缘计算(MEC)技术协助矿山物联网设备任务计算，同时利用能量收集技术为能量受限的矿山物联网设备供电。基于Q-learning的智能计算卸载机制实现在不可精确获取矿井系统模型的情况下动态探索最优计算卸载策略。此外，为处理复杂矿井环境下的维度灾难问题并减小策略离散化导致的离散化误差，该文还提出一种基于深度确定性策略梯度(DDPG)的计算卸载机制来进一步提高井下任务计算卸载性能。理论分析与仿真结果表明所提机制可降低系统的能量损耗、计算时延和任务处理失败率，有助于保障矿山物联网的安全和高效生产。
-
关键词:
- 矿山物联网 /
- 移动边缘计算 /
- 能量收集 /
- Q-learning /
- 深度确定性策略梯度(DDPG)
Abstract: This paper proposes an Energy Harvesting (EH)-assisted mining IoT intelligent computation offloading method for the mine IoT device with limited computing, energy, and memory resources and smart mining scenario with a large number of latency-sensitive computation tasks. Mobile Edge Computing (MEC) technology is used to assist task computing of mine IoT devices, and EH technology is investigated to power energy-constrained mine IoT devices. The intelligent computation offloading mechanism based on Q-learning can dynamically explore and optimize computation offloading policy under the condition of an unknown precise mine system model. In addition, a computation offloading mechanism based on Deep Deterministic Policy Gradient (DDPG) is proposed. The curse of dimensionality in complex mining scenarios is resolved, the discretization error caused by policy discretization is reduced, and the computation offloading performance is further improved. Theoretical analysis and simulation results verify that the proposed mechanism can reduce energy consumption, computing delays, and task failure rate. This helps ensure safety and improve the production efficiency of IoT in mining. -
算法1 基于Q-learning的计算卸载机制 初始化系统参数$\alpha $, $\gamma $, $B_1^0, \cdots ,B_M^0$ ,${g^{(1)}}$, ${b^0}$；设置学习迭代次数 1: For $k = 1,2,3, \cdots $do 2: 预测能量收集的产能${g^{(k)}}$，观察得到${b^{(k)}}$和$B_1^{(k - 1)}, \cdots ,B_M^{(k - 1)}$ 3: 得到状态${{\boldsymbol{s}}^{(k)} } = [B_1^{(k - 1)}, \cdots ,B_M^{(k - 1)},{g^{(k)} },{b^{(k)} }]$ 4: 基于$\varepsilon {\text{ - greedy}}$选择计算卸载策略${{\boldsymbol{a}}^{(k)} } = [{i^{(k)} },{x^{(k)} }]$，并卸载任务量$ {R^{(k)}}{x^{(k)}} $至边缘服务器$i$ 5: 与环境交互，得到电量变化${b^{(k + 1)}}{\text{ = }}\max \{ {b^{(k)}} - {E^{(k)}} + {g^{(k)}},0\} $，计算评估${E^{(k)}}$, ${T^{(k)}}$和${I}({b^{(k + 1)} } = 0)$ 6: 通过式(7)获得效益${U^{(k)}}$，通过式(8)更新Q值$Q({{\boldsymbol{s}}^{(k)} },{{\boldsymbol{a}}^{(k)} })$ 7: End 
下载: 导出CSV
算法2 基于DDPG的计算卸载机制 初始化学习参数$\alpha $,$\lambda $,${\xi _1}$,${\xi _2}$,$Z$,$\kappa $; OU噪声$\mathcal{N}$的参数；设置学习迭代次数 1: 观测得到${g^{(k)}}$, ${b^{(k)}}$, $B_1^{(k - 1)}, \cdots ,B_M^{(k - 1)}$，组成状态$ {s^{(k)}} = [B_1^{(k - 1)}, \cdots ,B_M^{(k - 1)},{g^{(k)}},{b^{(k)}}] $ 2: For $k = 1,2,3, \cdots $ do 3: 基于状态$ {s^{(k)}} $输出确定性动作，加入噪声$\mathcal{N}$后，产生卸载策略${{\boldsymbol{a}}^{(k)} } = \mu ({s^{(k)} }|{\xi _2}^{(k)}) + \mathcal{N}$ 4: 卸载任务量$ {R^{(k)}}{x^{(k)}} $至边缘服务器$i$，评估得到${s^{(k + 1)}}$和${U^{(k)}}$ 5: 将$({{\boldsymbol{s}}^{(k)} },{{\boldsymbol{a}}^{(k)} },{U^{(k)} },{s^{(k + 1)} })$存入经验池 6: 从经验池中抽取$Z$组学习经验$({s_h},{a_h},{U_h},{s_{h + 1}}),h \in [1,Z]$，更新网络参数 7: 由式(10)和式(11)更新Critic网络参数${\xi _1}$，由式(12)更新Actor网络参数${\xi _2}$ 8: 由式(13)对Target网络参数${\xi _1}^\prime ,{\xi _2}^\prime $进行软更新 9: ${{\boldsymbol{s}}^{(k)} }{\text{ = } }{{\boldsymbol{s}}^{(k + 1)} }$ 10: End for
下载: 导出CSV
表 1 DDRLOM机制的超参数设置
参数 数值 Actor/Critic学习率 0.0001/0.002 折扣因子 0.9 OU噪声均值/标准差/均值回归速度 5/0.015/40 batch-size 64 激活函数 Leaky ReLU 卷积网络隐藏层数/隐藏的单元数 2层/32,16
下载: 导出CSV
-
[1] 丁恩杰, 俞啸, 夏冰, 等. 矿山信息化发展及以数字孪生为核心的智慧矿山关键技术[J]. 煤炭学报, 2022, 47(1): 564–578. doi: 10.13225/j.cnki.jccs.yg21.1930DING Enjie, YU Xiao, XIA Bing, et al. Development of mine informatization and key technologies of intelligent mines[J]. Journal of China Coal Society, 2022, 47(1): 564–578. doi: 10.13225/j.cnki.jccs.yg21.1930 [2] MENG Yifan and LI Jingzhao. Task offloading and resource allocation mechanism of moving edge computing in mining environment[J]. IEEE Access, 2021, 9: 155534–155542. doi: 10.1109/ACCESS.2021.3129464 [3] 赵小虎, 张凯, 赵志凯, 等. 矿山物联网网络技术发展趋势与关键技术[J]. 工矿自动化, 2018, 44(4): 1–7. doi: 10.13272/j.issn.1671-251x.17324ZHAO Xiaohu, ZHANG Kai, ZHAO Zhikai, et al. Developing trend and key technologies of network technology of mine Internet of things[J]. Industry and Mine Automation, 2018, 44(4): 1–7. doi: 10.13272/j.issn.1671-251x.17324 [4] DENG Xiaoheng, YIN Jian, GUAN Peiyuan, et al. Intelligent delay-aware partial computing task offloading for multiuser industrial Internet of Things through edge computing[J]. IEEE Internet of Things Journal, 2023, 10(4): 2954–2966. doi: 10.1109/JIOT.2021.3123406 [5] PORAMBAGE P, OKWUIBE J, LIYANAGE M, et al. Survey on multi-access edge computing for internet of things realization[J]. IEEE Communications Surveys & Tutorials, 2018, 20(4): 2961–2991. doi: 10.1109/COMST.2018.2849509 [6] SUN Lu, WANG Jie, and LIN Bin. Task allocation strategy for MEC-enabled IIoTs via Bayesian network based evolutionary computation[J]. IEEE Transactions on Industrial Informatics, 2021, 17(5): 3441–3449. doi: 10.1109/TII.2020.3019572 [7] DINH T Q, TANG Jianhua, LA Q D, et al. Offloading in mobile edge computing: Task allocation and computational frequency scaling[J]. IEEE Transactions on Communications, 2017, 65(8): 3571–3584. doi: 10.1109/TCOMM.2017.2699660 [8] 袁亮, 俞啸, 丁恩杰, 等. 矿山物联网人-机-环状态感知关键技术研究[J]. 通信学报, 2020, 41(2): 1–12. doi: 10.11959/j.issn.1000-436x.2020036YUAN Liang, YU Xiao, DING Enjie, et al. Research on key technologies of human-machine-environment states perception in mine Internet of Things[J]. Journal on Communications, 2020, 41(2): 1–12. doi: 10.11959/j.issn.1000-436x.2020036 [9] 周代勇. 井下风致振动压电能量收集技术[J]. 煤矿安全, 2021, 52(9): 153–156. doi: 10.13347/j.cnki.mkaq.2021.09.024ZHOU Daiyong. Wind-induced vibration piezoelectric energy harvesting technology in underground mine[J]. Safety in Coal Mines, 2021, 52(9): 153–156. doi: 10.13347/j.cnki.mkaq.2021.09.024 [10] YE Junliang and GHARAVI H. Deep reinforcement learning-assisted energy harvesting wireless networks[J]. IEEE Transactions on Green Communications and Networking, 2021, 5(2): 990–1002. doi: 10.1109/TGCN.2020.3045075 [11] RODRIGUEZ J C, NICO V, and PUNCH J. Powering wireless sensor nodes for industrial IoT applications using vibration energy harvesting[C]. IEEE 5th World Forum on Internet of Things, Limerick, Ireland, 2019: 392–397. [12] SUN Yingying, SONG Chunhe, YU Shimao, et al. Energy-efficient task offloading based on differential evolution in edge computing system with energy harvesting[J]. IEEE Access, 2021, 9: 16383–16391. doi: 10.1109/ACCESS.2021.3052901 [13] RANJAN A, SAHU H B, and MISRA P. Wave propagation model for wireless communication in underground mines[C]. 2015 IEEE Bombay Section Symposium (IBSS), Mumbai, India, 2015: 1–5. [14] LEI Lei, XU Huijuan, XIONG Xiong, et al. Multiuser resource control with deep reinforcement learning in IoT edge computing[J]. IEEE Internet of Things Journal, 2019, 6(6): 10119–10133. doi: 10.1109/JIOT.2019.2935543 [15] XIAO Liang, LU Xiaozhen, XU Tangwei, et al. Reinforcement learning-based mobile offloading for edge computing against jamming and interference[J]. IEEE Transactions on Communications, 2020, 68(10): 6114–6126. doi: 10.1109/TCOMM.2020.3007742 [16] MIN Minghui, WAN Xiaoyue, XIAO Liang, et al. Learning-based privacy-aware offloading for healthcare IoT with energy harvesting[J]. IEEE Internet of Things Journal, 2019, 6(3): 4307–4316. doi: 10.1109/JIOT.2018.2875926 [17] WANG Hao and HUANG Jianwei. Incentivizing energy trading for interconnected microgrids[J]. IEEE Transactions on Smart Grid, 2018, 9(4): 2647–2657. doi: 10.1109/TSG.2016.2614988 [18] MIN Minghui, XIAO Liang, CHEN Ye, et al. Learning-based computation offloading for IoT devices with energy harvesting[J]. IEEE Transactions on Vehicular Technology, 2019, 68(2): 1930–1941. doi: 10.1109/TVT.2018.2890685 [19] WANG Jiadai, ZHAO Lei, LIU Jiajia, et al. Smart resource allocation for mobile edge computing: A deep reinforcement learning approach[J]. IEEE Transactions on Emerging Topics in Computing, 2021, 9(3): 1529–1541. doi: 10.1109/TETC.2019.2902661 [20] QIU Chengrun, HU Yang, CHEN Yan, et al. Deep deterministic policy gradient (DDPG)-based energy harvesting wireless communications[J]. IEEE Internet of Things Journal, 2019, 6(5): 8577–8588. doi: 10.1109/JIOT.2019.2921159 [21] REN Jieying and XU Shaoyi. DDPG based computation offloading and resource allocation for MEC systems with energy harvesting[C]. 2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring), Helsinki, Finland, 2021: 1–5. [22] MAO Yuyi, ZHANG Jun, and LETAIEF K B. Dynamic computation offloading for mobile-edge computing with energy harvesting devices[J]. IEEE Journal on Selected Areas in Communications, 2016, 34(12): 3590–3605. doi: 10.1109/JSAC.2016.2611964 [23] QIN Meng, CHENG Nan, JING Zewei, et al. Service-oriented energy-latency tradeoff for IoT task partial offloading in MEC-enhanced multi-RAT networks[J]. IEEE Internet of Things Journal, 2021, 8(3): 1896–1907. doi: 10.1109/JIOT.2020.3015970 [24] 国家市场监督管理总局, 国家标准化管理委员会. GB/T 3836.1-2021 爆炸性环境 第1部分: 设备 通用要求[S]. 北京: 中国标准出版社, 2021.State Administration for Market Regulation and Standardization Administration of the People's Republic of China. GB/T 3836.1-2021 Explosive atmospheres—Part 1: Equipment—General requirements[S]. Beijing: Standards Press of China, 2021. -
下载:
图(6) / 表(3)
计量
- 文章访问数: 283
- HTML全文浏览量: 128
- PDF下载量: 74
- 被引次数: 0
