Network Selection Algorithm Based on Improved Deep Q-Learning

MA Bin; CHEN Haibo; ZHANG Chao

doi:10.11999/JEIT200930

Volume 44 Issue 1

Jan. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(1): 346-353

MA Bin, CHEN Haibo, ZHANG Chao. Network Selection Algorithm Based on Improved Deep Q-Learning[J]. Journal of Electronics & Information Technology, 2022, 44(1): 346-353. doi: 10.11999/JEIT200930

Citation:

MA Bin, CHEN Haibo, ZHANG Chao. Network Selection Algorithm Based on Improved Deep Q-Learning[J]. Journal of Electronics & Information Technology, 2022, 44(1): 346-353. doi: 10.11999/JEIT200930

MA Bin, CHEN Haibo, ZHANG Chao. Network Selection Algorithm Based on Improved Deep Q-Learning[J]. Journal of Electronics & Information Technology, 2022, 44(1): 346-353. doi: 10.11999/JEIT200930

Citation:

MA Bin, CHEN Haibo, ZHANG Chao. Network Selection Algorithm Based on Improved Deep Q-Learning[J]. Journal of Electronics & Information Technology, 2022, 44(1): 346-353. doi: 10.11999/JEIT200930

PDF( 1916 KB)

Network Selection Algorithm Based on Improved Deep Q-Learning

doi: 10.11999/JEIT200930

1.
Chongqing Key Laboratory of Computer Network and Communication Technology, Chongqing University of Post and Telecommunications, Chongqing 400065, China
2.
Institute of Computer Science and Technology, Chongqing University of Post and Telecommunications, Chongqing 400065, China

Funds: The Major Project of Science and Technology Research of Chongqing Education Commission (KJZD-M201900602), The Key Project of Science and Technology Research of Chongqing Education Commission (KJZD-M201800603), The Foundation Research and Advanced Exploration Project of Chongqing (CSTC2018jcyjAX0432), The Project of Science Research Innovation of Chongqing Graduate Students (CYS20256)

Received Date: 2020-10-30
Rev Recd Date: 2021-05-26

Available Online: 2021-08-24

Publish Date: 2022-01-10

Abstract

Abstract

In ultra dense heterogeneous wireless network with sleep mechanism, in view of the problem that the network dynamic is enhanced and the handoff performance is reduced, a network selection algorithm based on improved deep Q-learning is proposed. Firstly, according to the dynamic analysis of the network, a deep Q-learning network selection model is constructed; Secondly, the training samples and weights of the offline training module in deep Q-learning network selection model, which are transferred to the online network decision-making module through the transfer learning; Finally, the training samples and weights of transfer are used to accelerate the process of training neural network, and the optimal network selection strategy is obtained. Experimental results demonstrate that the proposed algorithm improves significantly the performance degradation of high dynamic network handoff caused by sleep mechanism and the time complexity of traditional deep Q-learning algorithm for online network selection.
- Ultra dense heterogeneous wireless network,
- Improved deep Q-learning,
- Network selection

FullText(HTML)

References(17)

References

[1]	YAN Xiaohuan, ŞEKERCIOĞLU A, and NARAYANAN S. A survey of vertical handover decision algorithms in Fourth Generation heterogeneous wireless networks[J]. Computer Networks, 2010, 54(11): 1848–1863. doi: 10.1016/j.comnet.2010.02.006
[2]	XIE Shengdong and WU Meng. Adaptive variable threshold vertical handoff algorithm[C]. 2008 International Conference on Neural Networks and Signal Processing, Nanjing, China, 2008: 366–369. doi: 10.1109/ICNNSP.2008.4590373.
[3]	LIU Min, LI Zhongcheng, and GUO Xiaobing. An efficient handoff decision algorithm for vertical handoff between WWAN and WLAN[J]. Journal of Computer Science and Technology, 2007, 22(1): 114–120. doi: 10.1007/s11390-007-9016-8
[4]	ZAHRAN A H, LIANG Ben, and SALEH A. Signal threshold adaptation for vertical handoff in heterogeneous wireless networks[J]. Mobile Networks and Applications, 2006, 11(4): 625–640. doi: 10.1007/s11036-006-7326-7
[5]	HAIDER A, GONDAL I, and KAMRUZZAMAN J. Dynamic dwell timer for hybrid vertical handover in 4G coupled networks[C]. 2011 IEEE 73rd Vehicular Technology Conference (VTC Spring), Budapest, Hungary, 2011: 1–5. doi: 10.1109/VETECS.2011.5956636.
[6]	马彬, 张文静, 谢显中. 面向终端个性化服务的模糊垂直切换算法[J]. 电子与信息学报, 2017, 39(6): 1284–1290. doi: 10.11999/JEIT160839 MA Bin, ZHANG Wenjing, and XIE Xianzhong. Individualization service oriented fuzzy vertical handover algorithm[J]. Journal of Electronics &Information Technology, 2017, 39(6): 1284–1290. doi: 10.11999/JEIT160839
[7]	ALSAMHI S H and RAJPUT N S. An intelligent hand-off algorithm to enhance quality of service in high altitude platforms using neural network[J]. Wireless Personal Communications, 2015, 82(4): 2059–2073. doi: 10.1007/s11277-015-2333-2
[8]	马彬, 李尚儒, 谢显中. 异构无线网络中基于人工神经网络的自适应垂直切换算法[J]. 电子与信息学报, 2019, 41(5): 1210–1216. doi: 10.11999/JEIT180534 MA Bin, LI Shangru, and XIE Xianzhong. An adaptive vertical handover algorithm based on artificial neural network in heterogeneous wireless networks[J]. Journal of Electronics &Information Technology, 2019, 41(5): 1210–1216. doi: 10.11999/JEIT180534
[9]	NURJAHAN, RAHMAN S, SHARMA T, et al. PSO-NF based vertical handoff decision for ubiquitous heterogeneous wireless network (UHWN)[C]. 2016 International Workshop on Computational Intelligence (IWCI), Dhaka, Bangladesh, 2016: 153–158. doi: 10.1109/IWCI.2016.7860357.
[10]	YANG Bingtao, WANG Xue, and QIAN Zhihong. A multi-armed bandit model-based vertical handoff algorithm for heterogeneous wireless networks[J]. IEEE Communications Letters, 2018, 22(10): 2116–2119. doi: 10.1109/LCOMM.2018.2861731
[11]	CHEN Jiamei, WANG Yao, LI Yufeng, et al. QoE-aware intelligent vertical handoff scheme over heterogeneous wireless access networks[J]. IEEE Access, 2018, 6: 38285–38293. doi: 10.1109/ACCESS.2018.2853730
[12]	HAN Zijun, LEI Tao, LU Zhaoming, et al. Artificial intelligence-based handoff management for dense WLANs: A deep reinforcement learning approach[J]. IEEE Access, 2019, 7: 31688–31701. doi: 10.1109/access.2019.2900445
[13]	ALJERI N and BOUKERCHE A. A two-tier machine learning-based handover management scheme for intelligent vehicular networks[J]. Ad Hoc Networks, 2019, 94: 101930. doi: 10.1016/j.adhoc.2019.101930
[14]	马彬, 王梦雪, 谢显中. 超密集异构无线网络中基于位置预测的切换算法[J]. 电子与信息学报, 2020, 42(12): 2899–2907. doi: 10.11999/JEIT190751 MA Bin, WANG Mengxue, and XIE Xianzhong. Handoff algorithm based on location prediction in ultra-dense heterogeneous wireless network[J]. Journal of Electronics &Information Technology, 2020, 42(12): 2899–2907. doi: 10.11999/JEIT190751
[15]	ZHANG Xuefei, XIE Yuxuan, CUI Yushan, et al. Multi-slot coverage probability and SINR-based handover rate analysis for mobile user in HetNet[J]. IEEE Access, 2018, 6: 17868–17879. doi: 10.1109/ACCESS.2018.2821761
[16]	SAAD H, MOHAMED A, and ELBATT T. A cooperative Q-learning approach for distributed resource allocation in multi-user femtocell networks[C]. 2014 IEEE Wireless Communications and Networking Conference (WCNC), Istanbul, Turkey, 2014: 1490–1495. doi: 10.1109/WCNC.2014.6952410.
[17]	SUN Yaohua, PENG Mugen, and MAO Shiwen. Deep reinforcement learning-based mode selection and resource management for green fog radio access networks[J]. IEEE Internet of Things Journal, 2019, 6(2): 1960–1971. doi: 10.1109/JIOT.2018.2871020