Network Selection Algorithm Based on Improved Deep Q-Learning
-
摘要: 在引入休眠机制的超密集异构无线网络中，针对网络动态性增强，导致切换性能下降的问题，该文提出一种基于改进深度Q学习的网络选择算法。首先，根据网络的动态性分析，构建深度Q学习选网模型；其次，将深度Q学习选网模型中线下训练模块的训练样本与权值，通过迁移学习，将其迁移到线上决策模块中；最后，利用迁移的训练样本及权值加速训练神经网络，得到最佳选网策略。实验结果表明，该文算法显著改善了因休眠机制导致的高动态性网络切换性能下降问题，同时降低了传统深度Q学习算法在线上选网过程中的时间复杂度。Abstract: In ultra dense heterogeneous wireless network with sleep mechanism, in view of the problem that the network dynamic is enhanced and the handoff performance is reduced, a network selection algorithm based on improved deep Q-learning is proposed. Firstly, according to the dynamic analysis of the network, a deep Q-learning network selection model is constructed; Secondly, the training samples and weights of the offline training module in deep Q-learning network selection model, which are transferred to the online network decision-making module through the transfer learning; Finally, the training samples and weights of transfer are used to accelerate the process of training neural network, and the optimal network selection strategy is obtained. Experimental results demonstrate that the proposed algorithm improves significantly the performance degradation of high dynamic network handoff caused by sleep mechanism and the time complexity of traditional deep Q-learning algorithm for online network selection.
-
表 1 候选网络的参数值
网络 接收信号强度(dBm) 路径损失(dB) 噪声偏差(dBm) 吞吐量(kbps) 负载量(个) MBS1 –85 48 6 1100 68 MBS2 –70 51 9 900 52 SBS1 –78 47 8 2700 16 SBS2 –72 53 8 2600 23 SBS3 –86 50 7 2900 25 SBS4 –95 49 6 3100 20 AP1 –60 45 9 4800 12 AP2 –75 43 6 6400 8 AP3 –71 47 7 5500 10 
下载: 导出CSV
-
[1] YAN Xiaohuan, ŞEKERCIOĞLU A, and NARAYANAN S. A survey of vertical handover decision algorithms in Fourth Generation heterogeneous wireless networks[J]. Computer Networks, 2010, 54(11): 1848–1863. doi: 10.1016/j.comnet.2010.02.006 [2] XIE Shengdong and WU Meng. Adaptive variable threshold vertical handoff algorithm[C]. 2008 International Conference on Neural Networks and Signal Processing, Nanjing, China, 2008: 366–369. doi: 10.1109/ICNNSP.2008.4590373. [3] LIU Min, LI Zhongcheng, and GUO Xiaobing. An efficient handoff decision algorithm for vertical handoff between WWAN and WLAN[J]. Journal of Computer Science and Technology, 2007, 22(1): 114–120. doi: 10.1007/s11390-007-9016-8 [4] ZAHRAN A H, LIANG Ben, and SALEH A. Signal threshold adaptation for vertical handoff in heterogeneous wireless networks[J]. Mobile Networks and Applications, 2006, 11(4): 625–640. doi: 10.1007/s11036-006-7326-7 [5] HAIDER A, GONDAL I, and KAMRUZZAMAN J. Dynamic dwell timer for hybrid vertical handover in 4G coupled networks[C]. 2011 IEEE 73rd Vehicular Technology Conference (VTC Spring), Budapest, Hungary, 2011: 1–5. doi: 10.1109/VETECS.2011.5956636. [6] 马彬, 张文静, 谢显中. 面向终端个性化服务的模糊垂直切换算法[J]. 电子与信息学报, 2017, 39(6): 1284–1290. doi: 10.11999/JEIT160839MA Bin, ZHANG Wenjing, and XIE Xianzhong. Individualization service oriented fuzzy vertical handover algorithm[J]. Journal of Electronics &Information Technology, 2017, 39(6): 1284–1290. doi: 10.11999/JEIT160839 [7] ALSAMHI S H and RAJPUT N S. An intelligent hand-off algorithm to enhance quality of service in high altitude platforms using neural network[J]. Wireless Personal Communications, 2015, 82(4): 2059–2073. doi: 10.1007/s11277-015-2333-2 [8] 马彬, 李尚儒, 谢显中. 异构无线网络中基于人工神经网络的自适应垂直切换算法[J]. 电子与信息学报, 2019, 41(5): 1210–1216. doi: 10.11999/JEIT180534MA Bin, LI Shangru, and XIE Xianzhong. An adaptive vertical handover algorithm based on artificial neural network in heterogeneous wireless networks[J]. Journal of Electronics &Information Technology, 2019, 41(5): 1210–1216. doi: 10.11999/JEIT180534 [9] NURJAHAN, RAHMAN S, SHARMA T, et al. PSO-NF based vertical handoff decision for ubiquitous heterogeneous wireless network (UHWN)[C]. 2016 International Workshop on Computational Intelligence (IWCI), Dhaka, Bangladesh, 2016: 153–158. doi: 10.1109/IWCI.2016.7860357. [10] YANG Bingtao, WANG Xue, and QIAN Zhihong. A multi-armed bandit model-based vertical handoff algorithm for heterogeneous wireless networks[J]. IEEE Communications Letters, 2018, 22(10): 2116–2119. doi: 10.1109/LCOMM.2018.2861731 [11] CHEN Jiamei, WANG Yao, LI Yufeng, et al. QoE-aware intelligent vertical handoff scheme over heterogeneous wireless access networks[J]. IEEE Access, 2018, 6: 38285–38293. doi: 10.1109/ACCESS.2018.2853730 [12] HAN Zijun, LEI Tao, LU Zhaoming, et al. Artificial intelligence-based handoff management for dense WLANs: A deep reinforcement learning approach[J]. IEEE Access, 2019, 7: 31688–31701. doi: 10.1109/access.2019.2900445 [13] ALJERI N and BOUKERCHE A. A two-tier machine learning-based handover management scheme for intelligent vehicular networks[J]. Ad Hoc Networks, 2019, 94: 101930. doi: 10.1016/j.adhoc.2019.101930 [14] 马彬, 王梦雪, 谢显中. 超密集异构无线网络中基于位置预测的切换算法[J]. 电子与信息学报, 2020, 42(12): 2899–2907. doi: 10.11999/JEIT190751MA Bin, WANG Mengxue, and XIE Xianzhong. Handoff algorithm based on location prediction in ultra-dense heterogeneous wireless network[J]. Journal of Electronics &Information Technology, 2020, 42(12): 2899–2907. doi: 10.11999/JEIT190751 [15] ZHANG Xuefei, XIE Yuxuan, CUI Yushan, et al. Multi-slot coverage probability and SINR-based handover rate analysis for mobile user in HetNet[J]. IEEE Access, 2018, 6: 17868–17879. doi: 10.1109/ACCESS.2018.2821761 [16] SAAD H, MOHAMED A, and ELBATT T. A cooperative Q-learning approach for distributed resource allocation in multi-user femtocell networks[C]. 2014 IEEE Wireless Communications and Networking Conference (WCNC), Istanbul, Turkey, 2014: 1490–1495. doi: 10.1109/WCNC.2014.6952410. [17] SUN Yaohua, PENG Mugen, and MAO Shiwen. Deep reinforcement learning-based mode selection and resource management for green fog radio access networks[J]. IEEE Internet of Things Journal, 2019, 6(2): 1960–1971. doi: 10.1109/JIOT.2018.2871020 -
下载:
点击查看大图
图(8) / 表(1)
计量
- 文章访问数: 996
- HTML全文浏览量: 576
- PDF下载量: 118
- 被引次数: 0
