面向高密度车间通信的能量特征图谱资源分配算法

邱恭安; 刘永生; 章国安; 刘敏

doi:10.11999/JEIT250004

面向高密度车间通信的能量特征图谱资源分配算法

doi: 10.11999/JEIT250004 cstr: 32379.14.JEIT250004

1.
南通大学信息科学技术学院南通 226019
2.
南通大学人工智能与计算机学院南通 226019

基金项目: 国家自然科学基金(62471258)

详细信息

作者简介:
邱恭安：男，教授，研究方向为智能通信理论、广域车联网通信技术

刘永生：男，硕士生，研究方向为人工智能技术

章国安：男，教授，研究方向为数字孪生和车联网通信理论

刘敏：男，副教授，研究方向为无线智能通信理论与技术

通讯作者:
刘敏　liuming@ntu.edu.cn

中图分类号: TN929.5
计量
- 文章访问数: 295
- HTML全文浏览量: 167
- PDF下载量: 30
- 被引次数: 0
出版历程
- 收稿日期: 2025-01-06
- 修回日期: 2025-04-15
- 网络出版日期: 2025-05-08
- 刊出日期: 2025-08-27

Energy Characteristic Map Based Resource Allocation Algorithm for High-density V2V Communications

1.
School of Information Science and Technology, Nantong University, Nantong 226019, China
2.
School of Artificial Intelligence and Computer Science, Nantong University, Nantong 226019, China

Funds: The National Natural Science Foundation of China (62471258)

摘要

摘要: 车联网拓扑的动态性和资源分配的随机性增大了竞争接入相同资源的碰撞概率，降低了频谱资源效率。该文基于高密度车辆运动位置的邻接稳定性，提出了应用深度强化学习算法的能量特征图谱资源分配算法。首先，应用资源感知过程测量值计算候选资源块的时隙接收信号强度指数和子载波信干噪比值，构建候选资源库的时频能量特征图谱。随后，将能量特征图谱输入构建的两层深度神经网络(DNN)，以系统吞吐量为奖励函数训练DNN权值系数矩阵，建立匹配车辆运动状态的双DQN智能体模型，并存储于车载用户终端(VUE)。当车间通信请求分配资源建立通信链路时，VUE将感知过程计算的接收信号强度指数和信干噪比值输入存储的主DQN模型，根据训练后的DNN权值系数矩阵为车间通信选择高质量资源。应用离散时间马尔可夫链推导了资源接入碰撞概率、链路失效率与能量特征指数间的表达式。在高密度车间通信中，所提出的算法提高了交通安全消息传播可靠性和频谱效率，降低了端到端传播时延。在车辆密度不超过160 veh/km时，提出算法的消息分组正确接收率超过95%、时延低于4 ms，有效资源利用率高于0.6，满足编队行驶等车联网应用的性能要求。
- 蜂窝车联网 /
- 资源分配 /
- 深度强化学习 /
- 能量特征图谱
Abstract: Objective In high density scenarios, the random resource selection method has limitations in handling the high access collision probability of traffic safety messages under the limited frequency resource. At the same time, the variable topology accompanied by high mobility increases the failure rate of Vehicle to Vehicle (V2V) links. However, the traffic safety messages with ultra-high reliability and ultra-low latency are very important to ensure traffic safety and road efficiency under the present scenarios. To address these challenges, integrating the energy characteristic parameters in sub-frames and sub-carriers into the resource block map has emerged as a promising approach. By incorporating the distributed V2V links and designing effective reward functions, it is possible to decrease the access collision probability and smooth the dynamics of variable topology while maintaining high resource efficiency, thereby better meeting the needs of dense traffic. This research offers an intelligent solution for resource allocation in Cellular Vehicle to Everything (C-V2X) and provides theoretical support for the coordinated access of limited frequency with diverse link quality. Methods Based on the sustainable adjacency among the neighborhood vehicles in high-density V2V communications, Energy Characteristic Map (ECM) based resource allocation algorithm is proposed using Deep Reinforcement Learning algorithm. The guidance logic of the ECM algorithm periodically renews the energy indicators of candidate resources to train the weight coefficient matrix of two-layer Deep Neural Network (DNN) based on the characteristic results within the sensing window. The algorithm is then used as the action space in double Deep Q-learning Network (DQN) agent to maximize the V2V throughput, which has a main DQN and a target DQN. The state space in the DQN model includes the energy indicators of candidate resources such as the Received Signal Strength Indicator (RSSI) in sub-frames and Signal-to-Interference plus Noise Ratio (SINR) in sub-carriers, along with dynamic factors like the relative position and speed of other vehicles. The reward function is crucial for ensuring the resource efficiency and performance of the safety messages during the resource blocks selection. It accounts for factors such as the bandwidth and SINR of V2V links rewards to optimize decision-making. Additionally, the discount factor determines the weight of future rewards, balancing the importance of immediate versus future rewards. A lower discount factor typically emphasizes immediate rewards, leading to frequently resource block reselection, while a higher discount factor enhances the robustness of occupied resource. Results and Discussions The ECM algorithm periodically renews the energy indicators of candidate resources based on the characteristic results within the sensing window, which then serves as the action space in the double DQN agent. By defining an appropriate reward function, the main DQN in double DQN agent is established to select the candidate resource with high energy indicators for V2V links. The numerical results (Eq.(11) and Eq.(15)) between the packet received ratio and the energy indicators are analyzed using the discrete-time Markov chains. Simulation results show that the end-to-end disseminating performance of safety messages under variable V2V distances, simulated on WiLabV2Xsim, are represented (Fig.6, Fig.7). The reliability, PRR, is more than 0.95 under less than 160 veh/km (the blue line), while the comparative PRR is more than 0.95 under less than 120 veh/km (the green line) and 90 veh/km (the red line), respectively (Fig.10). At the same time, the latency, TD, is less than 3 ms under less than 180 veh/km (the blue line), while the comparative TD is less than 3 ms under less than 160 veh/km (the green line) and about 80 veh/km (the red line), respectively (Fig.11). The resource utilization, RU, is more than 0.6 under less than 180 veh/km (the blue line), while the comparative RU is more than 0.6 under less than 160 veh/km (the green line) and about 80 veh/km (the red line), respectively (Fig.12), demonstrating a 10～20% improvement in resource efficiency. When the discount factor is set to 0.9 while the learning rate is set to 0.01 (Fig.8, Fig.9), the VUE selects the resource blocks that balances immediate and long-term throughput, effectively improving the robustness of the main DQN, which meets the advanced V2V service requirements such as platooning in C-V2X. Conclusions This paper addresses the challenge of resource allocation in high-density V2V communications by integrating the ECM algorithm with double DQN agent. The proposed resource selection scheme enhances the RSS algorithm by establishing distributed V2V links using high quality resource blocks to maximize throughput. The scheme is evaluated through disseminating safety messages simulations under variable density, and the results show that: (1) The proposed scheme has high reliability with more than 0.95 PRR and ultra-low latency with less than 3 ms TD under upper 160 veh/km. (2) The resource efficiency has been improved by 10～20% over the RSS method; (3) Long-term and short-term rewards are considered by selecting the discount factor of 0.9 and the learning rate of 0.01 and enhance the robustness of DQN model. However, this study has not considered different resource characteristics for the heterogeneous messages with diverse Quality of Service (QoS) providing, which should be accounted for in future work.
- C-V2X /
- Resource allocation /
- Deep reinforcement learning /
- Energy characteristic map

HTML全文

图 1 密集交通应用场景

下载: 全尺寸图片幻灯片

图 2 能量特征图谱模型

下载: 全尺寸图片幻灯片

图 3 V2V通信的DRL模型

下载: 全尺寸图片幻灯片

图 4 DNN网络

下载: 全尺寸图片幻灯片

图 5 V2V通信DTMC分析模型

下载: 全尺寸图片幻灯片

图 6 ECM算法PRR与车辆密度关系曲线

下载: 全尺寸图片幻灯片

图 7 ECM算法TD与车辆密度关系曲线

下载: 全尺寸图片幻灯片

图 8 学习率和折扣因子对PRR影响曲线

下载: 全尺寸图片幻灯片

图 9 学习率和折扣因子对TD影响曲线

下载: 全尺寸图片幻灯片

图 10 不同算法的PRR与车辆密度关系曲线

下载: 全尺寸图片幻灯片

图 11 不同算法的TD与车辆密度关系曲线

下载: 全尺寸图片幻灯片

图 12 不同算法的RU与车辆密度关系曲线

下载: 全尺寸图片幻灯片

表 1 仿真参数

参数	参数值
信道带宽(MHz)	10
消息分组大小(Byte)	300
资源预留间隔(s)	0.1
资源保留概率	0.4
子载波间隔(kHz)	15
MCS	11
信道传播模型	WINNER+B1
学习率α	0.01
折扣因子γ	0.9
小批量经验样本数目	64

下载: 导出CSV

参考文献(18)

[1]	GU Bo, CHEN Weixiang, ALAZAB M, et al. Multiagent reinforcement learning-based semi-persistent scheduling scheme in C-V2X mode 4[J]. IEEE Transactions on Vehicular Technology, 2022, 71(11): 12044–12056. doi: 10.1109/TVT.2022.3189019.
[2]	SEIFHASHEMI F, HAJRASOULIHA A, and GHAHFAROKHI B S. Resource-aware multi-hop routing protocol for unicast cellular V2V communications[J]. IEEE Access, 2025, 13: 6584–6593. doi: 10.1109/ACCESS.2025.3526697.
[3]	GARCIA M H C, MOLINA-GALAN A, BOBAN M, et al. A tutorial on 5G NR V2X communications[J]. IEEE Communications Surveys & Tutorials, 2021, 23(3): 1972–2026. doi: 10.1109/COMST.2021.3057017.
[4]	王巨震, 江昊, 陈琪美, 等. C-V2X资源分配方法研究综述[J]. 太赫兹科学与电子信息学报, 2022, 20(1): 1–7. doi: 10.11805/tkyda2021145. WANG Juzhen, JIANG Hao, CHEN Qimei, et al. Summary of research on C-V2X resource allocation method[J]. Journal of Terahertz Science and Electronic Information Technology, 2022, 20(1): 1–7. doi: 10.11805/tkyda2021145.
[5]	JEON Y and KIM H. An explicit reservation-augmented resource allocation scheme for C-V2X sidelink mode 4[J]. IEEE Access, 2020, 8: 147241–147255. doi: 10.1109/ACCESS.2020.3015549.
[6]	李一兵, 王宁馨, 吕威. 蜂窝车联网中基于服务异构性的V2V通信资源分配算法研究[J]. 电子与信息学报, 2023, 45(1): 235–242. doi: 10.11999/JEIT211160. LI Yibing, WANG Ningxin, and LÜ Wei. Research on resource allocation algorithm based on service heterogeneity in V2V communication in C-V2X[J]. Journal of Electronics & Information Technology, 2023, 45(1): 235–242. doi: 10.11999/JEIT211160.
[7]	BANITALEBI N, AZMI P, MOKARI N, et al. Distributed learning-based resource allocation for self-organizing C-V2X communication in cellular networks[J]. IEEE Open Journal of the Communications Society, 2022, 3: 1719–1736. doi: 10.1109/OJCOMS.2022.3211340.
[8]	PARVINI M, SCHULZ P, and FETTWEIS G. Resource allocation in V2X networks: From classical optimization to machine learning-based solutions[J]. IEEE Open Journal of the Communications Society, 2024, 5: 1958–1974. doi: 10.1109/OJCOMS.2024.3380509.
[9]	陈发堂, 张若凡. 可重构智能反射面辅助的车联网资源分配算法研究[J]. 通信学报, 2023, 44(9): 70–78. doi: 10.11959/j.issn.1000−436x.2023145. CHEN Fatang and ZHANG Ruofan. Research on IoV resource allocation algorithm assisted by reconfigurable intelligent surface[J]. Journal on Communications, 2023, 44(9): 70–78. doi: 10.11959/j.issn.1000−436x.2023145.
[10]	许耀华, 王慧平, 王贵竹, 等. 基于图着色和三维匹配的车联网资源分配算法[J]. 系统工程与电子技术, 2023, 45(3): 869–875. doi: 10.12305/j.issn.1001-506X.2023.03.29. XU Yaohua, WANG Huiping, WANG Guizhu, et al. Resource allocation algorithm for internet of vehicles based on graph coloring and three-dimensional matching[J]. Systems Engineering and Electronics, 2023, 45(3): 869–875. doi: 10.12305/j.issn.1001-506X.2023.03.29.
[11]	JI Baofeng, DONG Bingyi, LI Da, et al. Optimization of resource allocation for V2X security communication based on multi-agent reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2025, 74(2): 1849–1861. doi: 10.1109/TVT.2023.3340424.
[12]	JI Maoxin, WU Qiong, FAN Pingyi, et al. Graph neural networks and deep reinforcement learning-based resource allocation for V2X communications[J]. IEEE Internet of Things Journal, 2025, 12(4): 3613–3628. doi: 10.1109/JIOT.2024.3469547.
[13]	YACHEUR B Y, AHMED T, and MOSBAH M. Efficient DRL-based selection strategy in hybrid vehicular networks[J]. IEEE Transactions on Network and Service Management, 2023, 20(3): 2400–2411. doi: 10.1109/TNSM.2023.3300653.
[14]	ZHANG Minglong, DOU Yi, MAROJEVIC V, et al. FAQ: A fuzzy-logic-assisted Q-learning model for resource allocation in 6G V2X[J]. IEEE Internet of Things Journal, 2024, 11(2): 2472–2489. doi: 10.1109/JIOT.2023.3294279.
[15]	LI Pengfei and HUANG Xinlin. Cooperative spectrum sensing approach in C-V2X based on multi-agent reinforcement learning[C]. Proceedings of 2023 17th International Conference on Telecommunications, Graz, Austria, 2023: 1–6. doi: 10.1109/ConTEL58387.2023.10199063.
[16]	WANG Junhan, HE He, CHA J, et al. Multi-agent reinforcement learning for efficient resource allocation in Internet of Vehicles[J]. Electronics, 2025, 14(1): 192. doi: 10.3390/electronics14010192.
[17]	WIJESIRI G P N B A, HAAPOLA J, and SAMARASINGHE T. A discrete-time Markov chain based comparison of the MAC layer performance of C-V2X Mode 4 and IEEE 802.11p[J]. IEEE Transactions of Communications, 2021, 69(4): 2505–2517. doi: 10.1109/TCOMM.2020.3044340.
[18]	LAGEN S, WANUGA K, ELKOTBY H, et al. New radio physical layer abstraction for system-level simulations of 5G networks[C]. Proceedings of the ICC 2020-2020 IEEE International Conference on Communications, Dublin, Ireland, 2020: 1–7. doi: 10.1109/ICC40277.2020.9149444.