Distributed Collaborative Path Planning Algorithm for Multiple Autonomous vehicles Based on Digital Twin

TANG Lun; DAI Jun; CHENG Zhangchao; ZHANG Hongpeng; CHEN Qianbin

doi:10.11999/JEIT230678

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2024 >

TANG Lun, DAI Jun, CHENG Zhangchao, ZHANG Hongpeng, CHEN Qianbin. Distributed Collaborative Path Planning Algorithm for Multiple Autonomous vehicles Based on Digital Twin[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT230678

Citation:

TANG Lun, DAI Jun, CHENG Zhangchao, ZHANG Hongpeng, CHEN Qianbin. Distributed Collaborative Path Planning Algorithm for Multiple Autonomous vehicles Based on Digital Twin[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT230678

Citation:

TANG Lun, DAI Jun, CHENG Zhangchao, ZHANG Hongpeng, CHEN Qianbin. Distributed Collaborative Path Planning Algorithm for Multiple Autonomous vehicles Based on Digital Twin[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT230678

PDF( 2263 KB)

Distributed Collaborative Path Planning Algorithm for Multiple Autonomous vehicles Based on Digital Twin

doi: 10.11999/JEIT230678

1.
School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
2.
Chongqing Key Laboratory of Mobile Communications Technology, Chongqing 400065, China

Funds: The National Natural Science Foundation of China (62071078), Sichuan and Chongqing Key R&D Projects (2021YFQ0053)

Received Date: 2023-07-07
Rev Recd Date: 2024-01-04

Available Online: 2024-01-29

Abstract

Abstract

Focusing on the problems of difficult cooperation between vehicles, low quality of the model trained by cooperation and poor effect of direct application of the obtained results to physical vehicles in the process of path planning for multiple Autonomous Vehicles (AVs), a distributed collaborative path planning algorithm is proposed for multiple AVs based on Digital Twin (DT). The algorithm is based on the Credibility-Weighted Decentralized Federated Reinforcement Learning (CWDFRL) to realize the path planning of multiple AVs. In this paper, the path planning problem of a single AVs is first modeled as the problem of minimizing the average task completion time under the constraints of driving behavior, which is transformed into Markov Decision Process (MDP) and solved by Deep Deterministic Policy Gradient algorithm (DDPG). Then Federated Learning (FL) is used to ensure the cooperation between vehicles. Aiming at the problem of low quality of global model update in centralized FL, this paper uses a decentralized FL training method based on dynamic node selection of reliability to improve the low quality. Finally, the DT is used to assist the training of the Decentralized Federated Reinforcement Learning (DFRL) model, and the trained model can be quickly deployed directly to the real-world AVs by taking advantage of the twin's ability of learning from DT environment. The simulation results show that compared with the existing methods, the proposed training framework can obtain a higher reward, effectively improve the utilization of the vehicle’s own speed, and at the same time reduce the average task completion time and collision probability of the vehicle swarm.
- Digital Twin (DT),
- Autonomous driving,
- Decentralized Federated Reinforcement Learning (DFRL),
- Path planning

FullText(HTML)

References(15)

References

[1]	KIRAN B R, SOBH I, TALPAERT V, et al. Deep reinforcement learning for autonomous driving: A survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(6): 4909–4926. doi: 10.1109/TITS.2021.3054625
[2]	LI Yanqiang, MING Yu, ZHANG Zihui, et al. An adaptive ant colony algorithm for autonomous vehicles global path planning[C]. 2021 IEEE 24th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Dalian, China, 2021: 1117–1122.
[3]	ZHOU Jian, ZHENG Hongyu, WANG Junmin, et al. Multiobjective optimization of lane-changing strategy for intelligent vehicles in complex driving environments[J]. IEEE Transactions on Vehicular Technology, 2020, 69(2): 1291–1308. doi: 10.1109/TVT.2019.2956504
[4]	ZHU Gongsheng, PEI Chunmei, DING Jiang, et al. Deep deterministic policy gradient algorithm based lateral and longitudinal control for autonomous driving[C]. 2020 5th International Conference on Mechanical, Control and Computer Engineering (ICMCCE), Harbin, China, 2020: 740–745.
[5]	SHI Dian, DING Jiahao, ERRAPOTU S M, et al. Deep Q-network-based route scheduling for TNC vehicles with passengers’ location differential privacy[J]. IEEE Internet of Things Journal, 2019, 6(5): 7681–7692. doi: 10.1109/JIOT.2019.2902815
[6]	KHALIL A A and RAHMAN M A. FED-UP: Federated deep reinforcement learning-based UAV path planning against hostile defense system[C]. 2022 18th International Conference on Network and Service Management (CNSM), Thessaloniki, Greece, 2022: 268–274.
[7]	LI Yijing, TAO Xiaofeng, ZHANG Xuefei, et al. Privacy-preserved federated learning for autonomous driving[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(7): 8423–8434. doi: 10.1109/TITS.2021.3081560
[8]	唐伦, 文明艳, 单贞贞, 等. 移动边缘计算辅助智能驾驶中基于高效联邦学习的碰撞预警算法[J]. 电子与信息学报, 2023, 45(7): 2406–2414. doi: 10.11999/JEIT220797 TANG Lun, WEN Mingyan, SHAN Zhenzhen et al. Collision warning algorithm based on efficient federated learning in mobile edge computing assisted intelligent driving[J]. Journal of Electronics & Information Technology, 2023, 45(7): 2406–2414. doi: 10.11999/JEIT220797
[9]	KARRAS A, KARRAS C, GIOTOPOULOS K C, et al. Peer to peer federated learning: Towards decentralized machine learning on edge devices[C]. 2022 7th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM), Ioannina, Greece, 2022: 1–9.
[10]	SHEN Gaoqing, LEI Lei, LI Zhilin, et al. Deep reinforcement learning for flocking motion of multi-UAV systems: Learn from a digital twin[J]. IEEE Internet of Things Journal, 2022, 9(13): 11141–11153. doi: 10.1109/JIOT.2021.3127873
[11]	GLAESSGEN E and STARGEL D. The digital twin paradigm for future NASA and U. S. air force vehicles[C]. 53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference, Honolulu, Hawaii, 2012: 1818.
[12]	TAO Fei, ZHANG He, LIU Ang, et al. Digital twin in industry: State-of-the-art[J]. IEEE Transactions on Industrial Informatics, 2019, 15(4): 2405–2415. doi: 10.1109/TII.2018.2873186
[13]	唐伦, 贺兰钦, 谭颀, 等. 基于深度确定性策略梯度的虚拟网络功能迁移优化算法[J]. 电子与信息学报, 2021, 43(2): 404–411. doi: 10.11999/JEIT190921 TANG Lun, HE Lanqin, TAN Qi, et al. Virtual network function migration optimization algorithm based on deep deterministic policy gradient[J]. Journal of Electronics & Information Technology, 2021, 43(2): 404–411. doi: 10.11999/JEIT190921
[14]	LIN Qifeng and LING Qing. Byzantine-robust federated deep deterministic policy gradient[C]. ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, Singapore, 2022: 4013–4017.
[15]	MA Xu, SUN Xiaoqian, WU Yuduo, et al. Differentially private byzantine-robust federated learning[J]. IEEE Transactions on Parallel and Distributed Systems, 2022, 33(12): 3690–3701. doi: 10.1109/TPDS.2022.3167434