面向智能网联汽车的安全多任务全景感知算法

黄晓舸; 陈名; 唐毅; 梁承超; 陈前斌

doi:10.11999/JEIT250749

面向智能网联汽车的安全多任务全景感知算法

doi: 10.11999/JEIT250749 cstr: 32379.14.JEIT250749

重庆邮电大学通信与信息工程学院重庆 400065

基金项目: 国家自然科学基金(62371082)，广西科技计划(AB24010317)，重庆市教育委员会科学技术研究项目(KJZD-K202400606)，重庆市科委自然科学基金(CSTB2023NSCQ-MSX0726, CSTB2023NSCQ-LZX0014)

详细信息

作者简介:
黄晓舸：女，博士，研究方向为移动通信技术、网络优化、区块链、物联网相关技术

陈名：男，硕士生，研究方向为移动通信技术、分布式学习、智能驾驶相关技术

唐毅：男，博士，高级工程师，研究方向为车路能、数字交通

梁承超：男，博士，教授，研究方向为无线通信、空天地一体化网络、(卫星)互联网架构与协议

陈前斌：男，博士，教授，研究方向为新一代移动通信网络、未来网络、LTE-Advanced异构小蜂窝网络

通讯作者:
黄晓舸　huangxg@cqupt.edu.cn

中图分类号: TN92
计量
- 文章访问数: 253
- HTML全文浏览量: 132
- PDF下载量: 30
- 被引次数: 0
出版历程
- 收稿日期: 2025-08-12
- 修回日期: 2026-04-12
- 录用日期: 2026-04-12
- 网络出版日期: 2026-04-30
- 刊出日期: 2026-06-15

Secure Multi-Task Federated Panoptic Perception Algorithm for Connected Autonomous Vehicles

School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Funds: The National Natural Science Foundation of China (62371082), Guangxi Science and Technology Project (AB24010317), The Science and Technology Project of Chongqing Education Commission (KJZD-K202400606), The Natural Science Foundation of Chongqing (CSTB2023NSCQ-MSX0726, CSTB2023NSCQ-LZX0014)

摘要

摘要: 随着车联网和深度学习技术的发展，智能网联汽车(Connected Autonomous Vehicles, CAV)能够收集驾驶场景中的图像数据，并借助卷积神经网络进行特征提取和处理，以实现对驾驶环境的高效感知。然而，由于驾驶场景的复杂性，单任务模型难以处理多样化的感知需求。此外，深度学习模型的优异表现依赖于海量数据，而单车收集的数据不足以训练出具有泛化性的模型。联邦学习技术打破了车辆间的信息孤岛，使车辆在不上传本地数据的情况下共享模型，通过上传本地模型至中心服务器进行模型聚合，从而获得性能更优的全局模型。因此，该文提出一种安全多任务全景感知算法。首先，构建了一个全景感知模型，使车辆能够并行执行驾驶场景中的多个感知任务，实现全景感知。其次，设计了一种基于混合评分的CAV选择算法，以筛选高质量的本地模型，最后，提出一种基于Shamir秘密共享的全局模型聚合算法，在聚合过程中采用秘密共享方法，避免因服务器遭受攻击或宕机导致的数据泄露。仿真结果验证了所提算法的有效性。
- 全景感知 /
- 深度学习 /
- 联邦学习 /
- 秘密共享
Abstract: Objective Connected autonomous vehicles (CAVs) need to perceive complex driving environments accurately and in real time, including traffic object detection, drivable-area segmentation, and lane-line detection. Existing single-task perception models are difficult to meet these diverse requirements simultaneously, while running multiple models in sequence increases latency and is unsuitable for safety-critical autonomous driving. In addition, the performance of deep learning models depends heavily on large-scale and diverse training data, whereas the data collected by an individual CAV are usually limited and scenario-specific. Federated learning (FL) provides a feasible way to train a more generalized global model without uploading raw vehicle data. However, conventional FL still faces several challenges in Internet of Vehicles scenarios, including malicious CAVs that upload low-quality or poisoned local models, privacy leakage from uploaded model parameters, and the single point of failure caused by centralized aggregation. To address these problems, this paper proposes a Secure Multi-task Federated Panoptic Perception (SMFPP) algorithm for connected autonomous vehicles. The goal is to achieve efficient multi-task driving perception, reliable collaborative model training, malicious-node resistance, privacy-preserving aggregation, and fault tolerance in a unified framework. Methods The proposed SMFPP framework consists of a cloud layer, an edge layer, and a terminal layer. The cloud layer publishes FL tasks and provides global coordination, the edge layer contains multiple road side units (RSUs) responsible for model aggregation, and the terminal layer consists of CAVs that train local models with their own driving-scene data. First, a YOLO-based multi-task panoptic perception model is designed as the local model. Its backbone network extracts visual features from vehicle images, while the spatial pyramid pooling fast module and feature pyramid network perform multi-scale feature fusion. Three task-specific detection heads are constructed to perform traffic object detection, drivable-area segmentation, and lane-line detection in parallel, thereby reducing perception latency and improving real-time applicability. The overall loss function combines the losses of the three tasks, including object detection losses and segmentation-related losses. Second, to defend against poisoning attacks and low-quality updates in FL, a hybrid-scoring-based CAV selection algorithm is proposed. For direct scoring, each CAV evaluates the cosine similarity between its local model and the previous global model, as well as the update age of its local model. For indirect scoring, RSUs select high-reputation CAVs to conduct encrypted cross-validation of other CAVs’ local models, and the model quality is evaluated according to local validation performance. The direct and indirect scores are normalized and used as the input of K-means clustering to select CAVs with high model quality, good timeliness, and strong consistency. The reputation value of each CAV is then updated for subsequent training rounds. Third, to avoid centralized aggregation risks, a Shamir-secret-sharing-based global model aggregation algorithm is developed. Each selected CAV splits its local model parameters into secret shares and sends them to different RSUs. Each RSU aggregates only the received shares and returns an aggregation share to CAVs. When a CAV receives at least the threshold number of aggregation shares, it reconstructs the global model by Lagrange interpolation. In this way, no single RSU can obtain a complete local model, and the system can still work when some RSUs fail. Results and Discussions Experiments are conducted on the BDD100K dataset, which contains diverse driving scenes from urban roads, suburban areas, and highways. In the basic simulation setting, five CAVs participate in FL training, each CAV uses 20% of the dataset to train its local model, and five RSUs serve as edge aggregation nodes. The effectiveness of the hybrid-scoring-based CAV selection algorithm is first verified under different numbers of malicious CAVs. As shown in Fig. 2, the mAP50 values of normal CAVs increase with training rounds and gradually converge, whereas malicious CAVs maintain low and unstable mAP50 values. Fig.3 further shows that a malicious CAV is excluded from global aggregation throughout training, and the global model achieves better convergence when the proposed selection algorithm is used. Compared with FedAvg, FedCS, and FedAsync, the proposed SMFPP framework obtains the best overall performance, achieving 86.4% Recall, 77.7% mAP50, 89.6% mIoU, 82.4% Accuracy, and 27.2% IoU, as reported in Table 1. For traffic object detection, SMFPP achieves 86.4% Recall and 77.7% mAP50, and its mAP50 is higher than those of MultiNet, DLT-Net, Fast R-CNN, YOLOv8n, YOLOP, A-YOLOM, and YOLOMH, as shown in Table 2. For drivable-area segmentation, SMFPP reaches 89.6% mIoU, outperforming MultiNet, DLT-Net, PSPNet, YOLOv8n(seg), YOLOP, A-YOLOM, and YOLOMH by 20.2%, 18.8%, 2.0%, 12.2%, 0.3%, 0.9%, and 0.4%, respectively, as shown in Table 3. For lane-line detection, SMFPP achieves the best Accuracy of 82.4% and IoU of 27.2%, as shown in Table 4. In both daytime and nighttime scenarios, SMFPP performs better than YOLOP on all evaluation metrics, demonstrating its adaptability to different illumination conditions (Table 5). In terms of efficiency, SMFPP contains only 3.45 million parameters and reaches 52.4 fps, indicating that it is lightweight and suitable for edge deployment (Table 6). The scalability experiment shows that, when the number of CAVs increases from 5 to 20, the global model performance improves consistently, with Recall, mAP50, mIoU, Accuracy, and IoU increasing to 88.6%, 79.5%, 90.4%, 85.2%, and 28.4%, respectively (Table 7). Moreover, the Shamir-secret-sharing-based aggregation scheme remains effective as long as at least three of the five RSUs survive. When the number of available RSUs is below the threshold, the global model cannot be correctly recovered; once enough RSUs recover, model training continues normally (Fig. 5). These results demonstrate that SMFPP improves perception accuracy, robustness against malicious CAVs, privacy protection, fault tolerance, and scalability. Conclusions This paper presents SMFPP, a secure multi-task federated panoptic perception algorithm for connected autonomous vehicles. The proposed method integrates a lightweight YOLO-based multi-task perception model with FL to solve the data-island problem among CAVs and to support collaborative training without exposing raw driving data. The hybrid-scoring-based CAV selection algorithm effectively filters malicious or low-quality local models by jointly considering model quality, update timeliness, and model similarity. The Shamir-secret-sharing-based aggregation scheme further protects local model parameters and removes the single point of failure in centralized FL aggregation. Simulation results on the BDD100K dataset verify that SMFPP achieves competitive or superior performance in object detection, drivable-area segmentation, and lane-line detection, while maintaining high inference efficiency and strong robustness. Future work may consider larger-scale heterogeneous vehicular networks, dynamic communication conditions, adaptive threshold settings, and real-world deployment in edge-assisted intelligent transportation systems.
- Panoptic perception /
- Deep learning /
- Federated learning /
- Secret sharing

HTML全文

图 1 基于联邦学习的CAV全景感知架构

下载: 全尺寸图片幻灯片

图 2 不同恶意CAV数量下本地模型的性能变化

下载: 全尺寸图片幻灯片

图 3 基于混合评分的CAV选择算法有效性验证

下载: 全尺寸图片幻灯片

图 4 模型在不同驾驶场景中的表现

下载: 全尺寸图片幻灯片

图 5 RSU存活数量对全局模型的影响

下载: 全尺寸图片幻灯片

1 基于混合评分的CAV选择算法

输入：CAV本地模型，CAV信誉值
输出：参与全局模型聚合的CAV集合
1：所有CAV在本地生成一对公私钥，私钥保存在本地，公钥广　　播发送给RSU
2：for $ {{v}}_{i} $ in $ {{V}}_{I} $ do
3：　RSU选择高信誉值CAV集合$ {{V}}_{K} $
4：　$ {{v}}_{i} $对本地模型加密并广播给RSU
5：　$ {{V}}_{K} $中的CAV获取该加密本地模型并使用解密
6：　使用本地数据测试解密后的本地模型
7：　计算$ {{v}}_{i} $本地模型的精度评分
8：end for
9：RSU计算所有CAV的间接评分
10：for $ {{v}}_{i} $ in $ {{V}}_{I} $ do
11：计算$ {{v}}_{i} $的本地模型年龄
12：计算$ {{v}}_{i} $的本地模型与上一轮全局模型的相似度
13：end for
14：使用K-means聚类算法进行CAV选择
15：更新CAV的信誉值

下载: 导出CSV

2 基于Shamir秘密共享的全局模型聚合算法

输入：CAV本地模型，RSU数量$ J $，CAV数量$ I $
输出：全局模型
1：for $ {v}_{i} $ $ i $ = 1:$ I $ do
2：　$ {v}_{i} $构建秘密多项式：
3：　$ {f}_{i}(x)={a}_{1,i}x+{a}_{2,i}{x}^{2}+\cdots +{a}_{t-1,i}{x}^{t-1}+{\boldsymbol{M}}_{i} $
4：　$ {{v}}_{i} $计算$ {f}_{i}(1),{f}_{i}(2),\cdots ,{f}_{i}(J) $并发送给$ J $个RSU
5：end for
6：for $ {{r}}_{j} $ $ j $ = 1:$ J $ do
7：　$ {{r}}_{j} $收到来自CAV的本地模型切片后，对其进行聚合
8：　聚合操作：$ {\boldsymbol{M}}_{j,\mathrm{mod}\text{el}}=\displaystyle\sum \nolimits_{i=1}^{I}\frac{1}{I}{f}_{i}(x) $
9：　$ {{r}}_{j} $将聚合结果发送给CAV
10：end for
11：CAV收到至少t个RSU的聚合结果后恢复出全局模型

下载: 导出CSV

表 1 全景感知模型在不同联邦学习框架下的性能

联邦学习框架	召回率(%)	mAP50(%)	mIoU(%)	准确率(%)	IoU(%)
FedAvg	82.4	74.3	87.4	80.3	25.3
FedCS	84.5	73.7	88.3	81.3	26.4
FedAsync	84.7	76.8	88.2	81.5	26.8
SMFPP	86.4	77.7	89.6	82.4	27.2
注：加粗数值为本文方法性能。

下载: 导出CSV

表 2 交通目标检测结果

模型	召回率(%)	mAP50(%)
MultiNet	80.5	57.4
DLT-Net	88.3	64.8
Fast R-CNN	80.9	61.2
YOLOv8n(det)	81.6	73.4
YOLOP	86.1	74.5
A-YOLOM	85.3	77.5
YOLOMH	86.6	76.2
SMFPP	86.4	77.7
注：加粗数值为表现最优数值。

下载: 导出CSV

表 3 可行驶区域分割结果

模型	mIoU(%)
MultiNet	69.4
DLT-Net	70.8
PSPNet	87.6
YOLOv8n(seg)	77.4
YOLOP	89.3
A-YOLOM	88.7
YOLOMH	89.2
SMFPP	89.6
注：加粗数值为表现最优数值。

下载: 导出CSV

表 4 车道线检测结果

模型	准确率(%)	IoU(%)
Enet	33.8	13.4
SCNN	34.6	14.9
Enet-SAD	35.3	15.2
YOLOv8n(seg)	80.7	20.4
YOLOP	81.8	24.9
A-YOLOM	82.0	26.5
YOLOMH	82.2	26.9
SMFPP	82.4	27.2
注：加粗数值为表现最优数值。

下载: 导出CSV

表 5 不同场景下模型性能

模型	场景	召回率(%)	mAP50(%)	mIoU(%)	准确率(%)	IoU(%)
YOLOP	日间场景	87.5	76.4	90.4	82.7	25.3
SMFPP	日间场景	88.4	78.6	91.8	84.2	28.2
YOLOP	夜间场景	85.3	74.3	88.3	80.5	24.2
SMFPP	夜间场景	85.7	76.4	89.6	81.3	26.5
注：加粗数值为本文方法性能。

下载: 导出CSV

表 6 模型参数与推理速度比较结果

模型	参数量(M)	推理速度(fps)
YOLOP	7.9	26.0
HybridNests	12.83	11.7
A-YOLOM	4.43	39.9
YOLOPv2	38.9	91.0
SMFPP	3.45	52.4

下载: 导出CSV

表 7 不同CAV规模下全局模型性能

CAV数量	召回率 (%)	mAP50 (%)	mIoU (%)	准确率 (%)	IoU (%)
5	86.4	77.7	89.6	82.4	27.2
10	87.8	79.2	89.8	84.1	27.5
20	88.6	79.5	90.4	85.2	28.4

下载: 导出CSV

参考文献(18)

[1]	DHINESH KUMAR R and RAMMOHAN A. Revolutionizing intelligent transportation systems with cellular vehicle-to-everything (C-V2X) technology: Current trends, use cases, emerging technologies, standardization bodies, industry analytics and future directions[J]. Vehicular Communications, 2023, 43: 100638. doi: 10.1016/j.vehcom.2023.100638.
[2]	HUANG Xiaoge, YIN Hongbo, CHEN Qianbin, et al. DAG-based swarm learning: A secure asynchronous learning framework for Internet of Vehicles[J]. Digital Communications and Networks, 2024, 10(6): 1611–1621. doi: 10.1016/j.dcan.2023.10.004.
[3]	HUANG Xiaoge, LI Wenjing, LIANG Chengchao, et al. Environment-aware personalized heterogeneous federated distillation for dual-layer blockchain-enabled internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2025, 74(12): 19552–19567. doi: 10.1109/TVT.2025.3586538.
[4]	潘仲夏, 沈丛麒, 骆汉光, 等. 面向车路协同的地理空间标识网络模态设计与场景应用[J]. 电子与信息学报, 2026, 48(2): 583–596. doi: 10.11999/JEIT250807. PAN Zhongxia, SHEN Congqi, LUO Hanguang, et al. Geospatial Identifier Network Modal Design and Scenario Applications for Vehicle-infrastructure Cooperative Networks[J]. Journal of Electronics & Information Technology, 2026, 48(2): 583–596. doi: 10.11999/JEIT250807.
[5]	GUO Yu, LIU R W, LU Yuxu, et al. Haze visibility enhancement for promoting traffic situational awareness in vision-enabled intelligent transportation[J]. IEEE Transactions on Vehicular Technology, 2023, 72(12): 15421–15435. doi: 10.1109/TVT.2023.3298041.
[6]	GIRSHICK R. Fast R-CNN[C]. IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 1440–1448. doi: 10.1109/ICCV.2015.169.
[7]	TERVEN J, CÓRDOVA-ESPARZA D M, and ROMERO-GONZÁLEZ J A. A comprehensive review of YOLO architectures in computer vision: From YOLOv1 to YOLOv8 and YOLO-NAS[J]. Machine Learning and Knowledge Extraction, 2023, 5(4): 1680–1716. doi: 10.3390/make5040083.
[8]	BADRINARAYANAN V, KENDALL A, and CIPOLLA R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481–2495. doi: 10.1109/TPAMI.2016.2644615.
[9]	ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]. IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 6230–6239. doi: 10.1109/CVPR.2017.660.
[10]	PAN Xingang, SHI Jianping, LUO Ping, et al. Spatial as deep: Spatial CNN for traffic scene understanding[C]. The 32th AAAI Conference on Artificial Intelligence, New Orleans, USA, 2018: 7276–7283. doi: 10.1609/aaai.v32i1.12301.
[11]	NEVEN D, DE BRABANDERE B, GEORGOULIS S, et al. Towards end-to-end lane detection: An instance segmentation approach[C]. IEEE Intelligent Vehicles Symposium, Changshu, China, 2018: 286–291. doi: 10.1109/IVS.2018.8500547.
[12]	TEICHMANN M, WEBER M, ZÖLLNER M, et al. Multinet: Real-time joint semantic reasoning for autonomous driving[C]. IEEE Intelligent Vehicles Symposium, Changshu, China, 2018: 1013–1020. doi: 10.1109/IVS.2018.8500504.
[13]	WU Dong, LIAO Manwen, ZHANG Weitian, et al. Correction to: YOLOP: You only look once for panoptic driving perception[J]. Machine Intelligence Research, 2023, 20(6): 952. doi: 10.1007/s11633-023-1452-6.
[14]	DAT V T, BAO N V H, and HUNG P D. HybridNets: End-to-end perception network[J]. Pattern Recognition and Image Analysis, 2025, 35(2): 106–118. doi: 10.1134/S1054661825700038.
[15]	XIAO Sa, HUANG Xiaoge, DENG Xuesong, et al. DB-FL: DAG blockchain-enabled generalized federated dropout learning[J]. Digital Communications and Networks, 2025, 11(3): 886–897. doi: 10.1016/j.dcan.2024.09.005.
[16]	郭显, 王典冬, 冯涛, 等. 基于同态加密的可验证隐私保护联邦学习方案[J]. 电子与信息学报, 2025, 47(4): 1113–1125. doi: 10.11999/JEIT240390. GUO Xian, WANG Diandong, FENG Tao, et al. A verifiable privacy protection federated learning scheme based on homomorphic encryption[J]. Journal of Electronics & Information Technology, 2025, 47(4): 1113–1125. doi: 10.11999/JEIT240390.
[17]	李亚红, 李一婧, 杨小东, 等. 基于同态加密和群签名的可验证联邦学习方案[J]. 电子与信息学报, 2025, 47(3): 758–768. doi: 10.11999/JEIT240796. LI Yahong, LI Yijing, YANG Xiaodong, et al. A verifiable federated learning scheme based on homomorphic encryption and group signature[J]. Journal of Electronics & Information Technology, 2025, 47(3): 758–768. doi: 10.11999/JEIT240796.
[18]	HU Xiaoya, LI Ruiqin, WANG Licheng, et al. A data sharing scheme based on federated learning in IoV[J]. IEEE Transactions on Vehicular Technology, 2023, 72(9): 11644–11656. doi: 10.1109/TVT.2023.3266100.