A Fast Deep Q-learning Network Edge Cloud Migration Strategy for Vehicular Service

Jun PENG; Chenglong WANG; Fu JIANG; Xin GU; Yueyue MU; Weirong LIU

doi:10.11999/JEIT190612

Volume 42 Issue 1

Jan. 2020

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2020 > 42(1): 58-64

Jun PENG, Chenglong WANG, Fu JIANG, Xin GU, Yueyue MU, Weirong LIU. A Fast Deep Q-learning Network Edge Cloud Migration Strategy for Vehicular Service[J]. Journal of Electronics & Information Technology, 2020, 42(1): 58-64. doi: 10.11999/JEIT190612

Citation:

Jun PENG, Chenglong WANG, Fu JIANG, Xin GU, Yueyue MU, Weirong LIU. A Fast Deep Q-learning Network Edge Cloud Migration Strategy for Vehicular Service[J]. Journal of Electronics & Information Technology, 2020, 42(1): 58-64. doi: 10.11999/JEIT190612

Citation:

PDF( 1450 KB)

A Fast Deep Q-learning Network Edge Cloud Migration Strategy for Vehicular Service

doi: 10.11999/JEIT190612

1.
School of Computer Science and Engineering, Central South University, Changsha 410083, China
2.
School of Automation, Central South University, Changsha 410083, China

Funds: The National Natural Science Foundation of China(61873353, 61672539)

Received Date: 2019-08-12
Rev Recd Date: 2019-11-04

Available Online: 2019-11-12

Publish Date: 2020-01-21

Abstract

Abstract

The high-speed movement of vehicles inevitably leads to frequent data migration between edge servers and increases communication delay, which brings great challenges to the real-time computing service of edge servers. To solve this problem, a real-time reinforcement learning method based on Deep Q-learning Networks according to vehicle motion Trajectory Process (DQN-TP) is proposed. The proposed algorithm separates the decision-making process from the training process by using two neural networks. The decision neural network obtains the network state in real time according to the vehicle’s movement track and chooses the migration method in the virtual machine migration and task migration. At the same time, the decision neural network uploads the decision records to the memory replay pool in the cloud. The evaluation neural network in the cloud trains with the records in the memory replay pool and periodically updates the parameters to the on-board decision neural network. In this way, training and decision-making can be carried out simultaneously. At last, a large number of simulation experiments show that the proposed algorithm can effectively reduce the latency compared with the existing methods of task migration and virtual machine migration.
- Intelligent transportation system,
- Virtual machine migration,
- Reinforcement learning,
- Deep Q-learning Networks(DQN)

FullText(HTML)

References(14)

References

ZHU Li, YU F R, WANG Yige, et al. Big data analytics in intelligent transportation systems: A survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 20(1): 383–398. doi: 10.1109/TITS.2018.2815678

D’OREY P M and FERREIRA M. ITS for sustainable mobility: A survey on applications and impact assessment tools[J]. IEEE Transactions on Intelligent Transportation Systems, 2014, 15(2): 477–493. doi: 10.1109/TITS.2013.2287257

彭军, 马东, 刘凯阳, 等. 基于LTE D2D技术的车联网通信架构与数据分发策略研究[J]. 通信学报, 2016, 37(7): 62–70. doi: 10.11959/j.issn.1000-436x.2016134

PENG Jun, MA Dong, LIU Kaiyang, et al. LTE D2D based vehicle networking communication architecture and data distributing strategy[J]. Journal on Communications, 2016, 37(7): 62–70. doi: 10.11959/j.issn.1000-436x.2016134

GAO Kai, HAN Farong, DONG Pingping, et al. Connected vehicle as a mobile sensor for real time queue length at signalized intersections[J]. Sensors, 2019, 19(9): 2059. doi: 10.3390/s19092059

KONG Yue, ZHANG Yikun, WANG Yichuan, et al. Energy saving strategy for task migration based on genetic algorithm[C]. 2018 International Conference on Networking and Network Applications, Xi’an, China, 2018: 330–336.

CHEN Xianfu, ZHANG Honggang, WU C, et al. Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning[J]. IEEE Internet of Things Journal, 2019, 6(3): 4005–4018. doi: 10.1109/JIOT.2018.2876279

SAHA S and HASAN M S. Effective task migration to reduce execution time in mobile cloud computing[C]. The 23rd International Conference on Automation and Computing, Huddersfield, UK, 2017: 1–5.

GONÇALVES D, VELASQUEZ K, CURADO M, et al. Proactive virtual machine migration in fog environments[C]. 2018 IEEE Symposium on Computers and Communications, Natal, Brazil, 2018: 742–745.

KIKUCHI J, WU C, JI Yusheng, et al. Mobile edge computing based VM migration for QoS improvement[C]. The 6th IEEE Global Conference on Consumer Electronics, Nagoya, Japan, 2017: 1–5.

CHOWDHURY M, STEINBACH E, KELLERER W, et al. Context-Aware task migration for HART-Centric collaboration over FiWi based tactile internet infrastructures[J]. IEEE Transactions on Parallel and Distributed Systems, 2018, 29(6): 1231–1246. doi: 10.1109/TPDS.2018.2791406

LU Wei, MENG Xianyu, and GUO Guanfei. Fast service migration method based on virtual machine technology for MEC[J]. IEEE Internet of Things Journal, 2019, 6(3): 4344–4354. doi: 10.1109/JIOT.2018.2884519

WANG Yanting, SHENG Min, WANG Xijun, et al. Mobile-edge computing: Partial computation offloading using dynamic voltage scaling[J]. IEEE Transactions on Communications, 2016, 64(10): 4268–4282. doi: 10.1109/TCOMM.2016.2599530

SUTTON R S and BARTO A G. Reinforcement Learning: An Introduction[M]. Cambridge: MIT Press, 1998: 25–42.

SNIA trace data[EB/OL]. http://iotta.snia.org/traces, 2018.

Relative Articles

Supplements(0)

Cited By

Proportional views