Hybrid Data Scheduling Method for Industrial Wireless Sensor Networks Based on Age of Information
-
摘要: 在工业无线传感器网络(IWSN)中，实时交付工业现场的周期性控制/传感数据流与非周期性事件数据流，是保障生产安全高效运行的关键。信息年龄(AoI)作为一种新兴的数据新鲜度衡量指标，能够从目标节点角度全面地度量网络数据交付的实时性。针对周期性和非周期性数据混合的工业无线传感器网络，该文在引入网络数据整体新鲜度指标的同时，考虑到周期性数据新鲜度在超过阈值后可能会对工业生产造成负面影响，建立了最小化系统平均AoI和周期性数据AoI逾期概率的联合优化模型，并将优化问题表述为马尔可夫决策过程(MDP)进行求解。由于传统基于相对值迭代的最优求解方法在大规模网络中因为维度灾难难以实施，因此采用深度强化学习(DRL)降低优化问题的状态空间维度，并改进决策探索机制以加快学习速度，提出了基于优化决策探索的深度强化学习(DRL-ODE)调度方法。仿真结果表明，所提方法能够提高网络数据交付的实时性，并有效减少周期性数据的AoI逾期概率。Abstract: In Industrial Wireless Sensor Networks (IWSN), timely delivery of periodic control/sensing data flows and aperiodic event data flows is crucial to ensure production safety and efficiency. As a new metric of data freshness, Age of Information (AoI) can comprehensively measure the real-time performance of data delivery from the perspective of destination node. For industrial wireless sensor networks with hybrid periodic and aperiodic data, the data freshness metric of whole network is introduced. Considering that the freshness of periodic data exceeding the threshold may have a negative impact on industrial production, a joint optimization model is established, which minimizes the system average AoI and the probability of AoI overdue for periodic data, and then the optimization problem is formulated as a Markov Decision Process (MDP). Since the traditional optimal solution method based on relative value iteration is difficult to implement in large-scale networks result from dimensional disasters, Deep Reinforcement Learning (DRL) is used to reduce the state space dimension of the optimization problem. Moreover, the decision exploration mechanism is improved to speed up the learning speed, and a scheduling method of deep reinforcement learning based on Optimal Decision Exploration (DRL-ODE) is proposed. Simulation results show that the proposed method can improve the timeliness of network data delivery while reducing the probability of AoI overdue for periodic data effectively.
-
算法1 DRL-ODE调度方法训练算法流程 (1) 初始化：网络参数${\mathbf{w}}$和${{\mathbf{w}}_N}$以及回放记忆单元 (2) for k = 0,1,···,K do (3) 生成一个0和1之间的随机数b； (4) if $b < \mu $ then (5) 根据式(12)计算每个源节点的期望下降值${e_m}$并生
成优化动作空间$O\left( k \right)$；(6) 从$O\left( k \right)$中随机选取决策$d\left( k \right)$； (7) else then (8) 选择$\min V\left( {s\left( k \right),d\left( k \right)|{\mathbf{w}}} \right)$对应的决策$d\left( k \right)$； (9) end (10) 当前值网络执行决策$d\left( k \right)$并与当前系统状态为$s\left( k \right)$的
IWSN环境交互；(11) 获取IWSN环境反馈的下一状态$s\left( {k + 1} \right)$和惩罚$c\left( k \right)$； (12) 将当前经验集合$\left( {s\left( k \right),d\left( k \right),c\left( k \right),s\left( {k + 1} \right)} \right)$存入回放
记忆单元；(13) 从回放记忆单元中随机选取经验集合并根据式(11)计算
损失函数$ L\left( {\mathbf{w}} \right) $；(14) 根据式(12)利用梯度下降法更新参数向量${\mathbf{w}}$； (15) 每迭代N次将当前值网络参数拷贝至目标值网络； (16) end for 
下载: 导出CSV
-
[1] 胡致远, 胡文前, 李香, 等. 面向业务可达性的广域工业互联网调度算法研究[J]. 电子与信息学报, 2021, 43(9): 2608–2616. doi: 10.11999/JEIT200583HU Zhiyuan, HU Wenqian, LI Xiang, et al. Research on wide area industrial internet scheduling algorithm based on service reachability[J]. Journal of Electronics &Information Technology, 2021, 43(9): 2608–2616. doi: 10.11999/JEIT200583 [2] SHA M, GUNATILAKA D, WU Chengjie, et al. Empirical study and enhancements of industrial wireless sensor–actuator network protocols[J]. IEEE Internet of Things Journal, 2017, 4(3): 696–704. doi: 10.1109/JIOT.2017.2653362 [3] 王恒, 朱元杰, 杨杭, 等. 基于优先级分类的工业无线网络确定性调度算法[J]. 自动化学报, 2020, 46(2): 373–384. doi: 10.16383/j.aas.c170722WANG Heng, ZHU Yuanjie, YANG Hang, et al. Deterministic scheduling algorithm with priority classification for industrial wireless networks[J]. Acta Automatica Sinica, 2020, 46(2): 373–384. doi: 10.16383/j.aas.c170722 [4] 段洁, 胡显静, 林欢, 等. 面向物联网数据特征的信息中心网络缓存方案[J]. 电子与信息学报, 2021, 43(8): 2240–2248. doi: 10.11999/JEIT200631DUAN Jie, HU Xianjing, LIN Huan, et al. Information-centric networking caching scheme for data characteristics of internet of things[J]. Journal of Electronics &Information Technology, 2021, 43(8): 2240–2248. doi: 10.11999/JEIT200631 [5] KAUL S, YATES R, and GRUTESER M. Real-time status: How often should one update?[C]. 2012 Proceedings IEEE INFOCOM, Orlando, USA, 2012: 2731–2735. [6] KAM C, KOMPELLA S, NGUYEN G D, et al. Effect of message transmission path diversity on status age[J]. IEEE Transactions on Information Theory, 2016, 62(3): 1360–1374. doi: 10.1109/TIT.2015.2511791 [7] KUANG Qiaobin, GONG Jie, CHEN Xiang, et al. Age-of-information for computation-intensive messages in mobile edge computing[C]. The 11th International Conference on Wireless Communications and Signal Processing (WCSP), Xi'an, China, 2019: 1–6. [8] KUANG Qiaobin, GONG Jie, CHEN Xiang, et al. Analysis on computation-intensive status update in mobile edge computing[J]. IEEE Transactions on Vehicular Technology, 2020, 69(4): 4353–4366. doi: 10.1109/TVT.2020.2974816 [9] KADOTA I, SINHA A, UYSAL-BIYIKOGLU E, et al. Scheduling policies for minimizing age of information in broadcast wireless networks[J]. IEEE/ACM Transactions on Networking, 2018, 26(6): 2637–2650. doi: 10.1109/TNET.2018.2873606 [10] YIN Bo, ZHANG Shuai, and CHENG Yu. Application-oriented scheduling for optimizing the age of correlated information: A deep-reinforcement-learning-based approach[J]. IEEE Internet of Things Journal, 2020, 7(9): 8748–8759. doi: 10.1109/JIOT.2020.2996562 [11] HSU Y P, MODIANO E, and DUAN Lingjie. Age of information: Design and analysis of optimal scheduling algorithms[C]. 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, Germany, 2017: 561–565. [12] TANG Haoyue, WANG Jintao, SONG Linqi, et al. Minimizing age of information with power constraints: Multi-user opportunistic scheduling in multi-state time-varying channels[J]. IEEE Journal on Selected Areas in Communications, 2020, 38(5): 854–868. doi: 10.1109/JSAC.2020.2980911 [13] 王恒, 段思勰, 谢鑫. 基于信息年龄优化的多信道无线网络调度方法[J]. 电子与信息学报, 2022, 44(2): 702–709. doi: 10.11999/JEIT210107WANG Heng, DUAN Sixie, and XIE Xin. Scheduling method for multi-channel wireless networks based on optimization of age of information[J]. Journal of Electronics &Information Technology, 2022, 44(2): 702–709. doi: 10.11999/JEIT210107 [14] BEDEWY A M, SUN Yin, and SHROFF N B. Optimizing data freshness, throughput, and delay in multi-server information-update systems[C]. 2016 IEEE International Symposium on Information Theory (ISIT), Barcelona, Spain, 2016: 2569–2573. [15] 赵悦超, 杨涛, 胡波. 无线传感器网络中基于信息年龄的状态更新策略[J]. 微电子学与计算机, 2020, 37(11): 29–34. doi: 10.19304/j.cnki.issn1000-7180.2020.11.006ZHAO Yuechao, YANG Tao, and HU Bo. A status updating policy based on age of information in wireless sensor network[J]. Microelectronics &Computer, 2020, 37(11): 29–34. doi: 10.19304/j.cnki.issn1000-7180.2020.11.006 [16] XIE Xin, WANG Heng, and WENG Mingjiang. A reinforcement learning approach for optimizing the age-of-computing-enabled IoT[J]. IEEE Internet of Things Journal, 2022, 9(4): 2778–2786. doi: 10.1109/JIOT.2021.3093156 [17] KASHEF M and MOAYERI N. Real-time scheduling for wireless networks with random deadlines[C]. The 13th International Workshop on Factory Communication Systems (WFCS), Trondheim, Norway, 2017: 1–9. [18] JIN Xi, KONG Fanxin, KONG Linghe, et al. A hierarchical data transmission framework for industrial wireless sensor and actuator networks[J]. IEEE Transactions on Industrial Informatics, 2017, 13(4): 2019–2029. doi: 10.1109/TII.2017.2685689 [19] XIE Xin, WANG Heng, YU Lei, et al. Online algorithms for optimizing age of information in the IoT systems with multi-slot status delivery[J]. IEEE Wireless Communications Letters, 2021, 10(5): 971–975. doi: 10.1109/LWC.2021.3052569 [20] LI Chengzhang, LI Shaoran, CHEN Yongce, et al. Minimizing age of information under general models for IoT data collection[J]. IEEE Transactions on Network Science and Engineering, 2020, 7(4): 2256–2270. doi: 10.1109/TNSE.2019.2952764 [21] KAM C, KOMPELLA S, NGUYEN G D, et al. On the age of information with packet deadlines[J]. IEEE Transactions on Information Theory, 2018, 64(9): 6419–6428. doi: 10.1109/TIT.2018.2818739 [22] 王恒, 陈鹏飞, 王平. 面向WIA-PA工业无线传感器网络的确定性调度算法[J]. 电子学报, 2018, 46(1): 68–74. doi: 10.3969/j.issn.0372-2112.2018.01.010WANG Heng, CHEN Pengfei, and WANG Ping. Deterministic scheduling algorithms for WIA-PA industrial wireless sensor networks[J]. Acta Electronica Sinica, 2018, 46(1): 68–74. doi: 10.3969/j.issn.0372-2112.2018.01.010 [23] BERTSEKAS D P. Dynamic Programming and Optimal Control[M]. 4th ed. Belmont: Athena Scientific, 2012. [24] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529–533. doi: 10.1038/nature14236 [25] ABD-ELMAGID M A, DHILLON H S, and PAPPAS N. A reinforcement learning framework for optimizing age of information in RF-powered communication systems[J]. IEEE Transactions on Communications, 2020, 68(8): 4747–4760. doi: 10.1109/TCOMM.2020.2991992 -
图(6) / 表(1)
计量
- 文章访问数: 697
- HTML全文浏览量: 525
- PDF下载量: 154
- 被引次数: 0
下载:
