TTSPD: A Multimodal Traffic Scene Perception Dataset Integrating Tire Data

YING Zongchen; GUI Lin; YANG Jiahan; ZHANG Fangwei; WANG Junfan; DONG Zhekang

doi:10.11999/JEIT260022

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

YING Zongchen, GUI Lin, YANG Jiahan, ZHANG Fangwei, WANG Junfan, DONG Zhekang. TTSPD: A Multimodal Traffic Scene Perception Dataset Integrating Tire Data[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260022

Citation:

YING Zongchen, GUI Lin, YANG Jiahan, ZHANG Fangwei, WANG Junfan, DONG Zhekang. TTSPD: A Multimodal Traffic Scene Perception Dataset Integrating Tire Data[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260022

Citation:

PDF( 3620 KB)

TTSPD: A Multimodal Traffic Scene Perception Dataset Integrating Tire Data

doi: 10.11999/JEIT260022 cstr: 32379.14.JEIT260022

YING Zongchen^{1, 2},
GUI Lin^{1, 2},
YANG Jiahan^{1, 2},
ZHANG Fangwei³,
WANG Junfan^{1, 2},
DONG Zhekang^{1, 2, 4
,
,}

1.
School of Electronic and Information, Hangzhou Dianzi University, Hangzhou 310018, China
2.
Zhejiang Key Laboratory of Intelligent Vehicle Electronics Research, Hangzhou 310018, China
3.
BH SENS (Shanghai) Electronics Co., Ltd, Shanghai 210619, China
4.
College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China

Funds: Yangtze River Delta Science and Technology Innovation Program under Grant No. YDZX20233100004028, Zhejiang Provincial Natural Science Foundation of China under Grant No. LZYQ25F020005

Accepted Date: 2026-02-05

Available Online: 2026-02-27

Abstract

Abstract

Objective With the rapid advancement of intelligent transportation systems (ITS) and autonomous driving technologies, accurate traffic environment perception has become a fundamental prerequisite for vehicle safety and decision-making. Current perception frameworks predominantly rely on high-resolution cameras and LiDAR sensors, which, although information-rich, introduce severe challenges across the Perception–Storage–Calculation pipeline. High acquisition costs limit large-scale deployment, while the massive data volume generated by high-dimensional sensors places substantial pressure on onboard storage and computational resources, often exceeding the power and thermal budgets of vehicle-grade edge platforms. These limitations motivate the exploration of alternative sensing paradigms that are cost-effective, compact, and computationally efficient, yet capable of maintaining high perception accuracy. In response, this study shifts the perception perspective from conventional external sensors to the tire–road contact interface, where rich physical interaction information is inherently embedded. The primary objective is to construct a novel multi-modal dataset, termed the Tire-integrated Traffic Scene Perception Dataset (TTSPD), which combines internal tire dynamics with external visual observations. Through this dataset, the study aims to investigate whether low-dimensional tire sensing data can complement or partially substitute high-dimensional visual data for accurate road surface classification, thereby establishing a new data morphology that better balances perception performance and system efficiency for future intelligent vehicles. Methods To construct a high-quality and practically usable multi-modal dataset, an integrated hardware–software acquisition framework was developed. From a hardware perspective, a specialized sensing system was designed by coupling tire-mounted multi-parameter sensors with a vehicle-mounted camera. To ensure reliable operation under the harsh mechanical conditions of a rotating tire, the sensing nodes were encapsulated using a rubber-based composite material, providing mechanical protection and long-term stability. Wireless data transmission was achieved using Bluetooth Low Energy (BLE) 5.0 with an adaptive frequency-hopping mechanism, enabling low-power and robust communication under high-speed rotation. During data acquisition, the system synchronously collected six types of internal tire signals, including radial acceleration, tire temperature, and tire pressure, yielding approximately 1.8 million sampling points. In parallel, a dashboard-mounted camera recorded high-resolution traffic scene imagery totaling 309 GB across four representative road surface conditions. To address the heterogeneity between high-frequency one-dimensional tire signals and two-dimensional visual data, a timestamp-based association strategy was adopted to perform scene-level temporal alignment, rather than enforcing strict frame-by-frame correspondence. Specifically, sensor sequences and image segments were grouped according to shared temporal windows and driving scenarios, ensuring semantic and temporal consistency at the scene scale. This alignment strategy reflects practical deployment conditions and forms the basis of the final TTSPD for multi-modal fusion research. Results and Discussions The effectiveness of the proposed TTSPD was validated through comprehensive road surface classification experiments using mainstream deep learning models. Initial evaluations based solely on visual data demonstrated strong baseline performance, with classification accuracies ranging from 87.25% to 93.75%(Table. 7), confirming the quality and diversity of the visual modality within the dataset. Beyond baseline validation, the core contribution of this study lies in quantifying the efficiency gains enabled by tire-based sensing. Comparative experiments were conducted by progressively reducing the amount of visual data while fusing low-dimensional tire signals, particularly radial acceleration(Table. 9). The results reveal that the multi-modal model achieves approximately 95% of the full-data baseline accuracy while using only about 38.75% of the original data volume. This substantial reduction in data dependency directly translates into notable system-level benefits. Specifically, storage requirements are reduced by approximately 61.25%, and overall model training time decreases by about 54.10%(Fig. 8). These findings indicate that tire dynamics encode high-value physical features related to road texture and surface conditions that are complementary to visual cues. Consequently, the proposed dataset supports the development of “lighter” perception pipelines without sacrificing recognition performance. Conclusions This study addresses the long-standing Perception–Storage–Calculation bottleneck in vision-dominated autonomous driving systems through the introduction of the Tire-integrated Traffic Scene Perception Dataset (TTSPD). By embedding multi-parameter sensors within tires using rubber-based encapsulation and enabling stable wireless transmission via BLE 5.0, a robust tire–camera data acquisition system was successfully realized. The resulting dataset spans four common and safety-critical road surface types—cement, asphalt, damaged, and water-covered roads—providing a comprehensive foundation for multi-modal perception research. Experimental results demonstrate that fusing low-dimensional tire sensing data with visual information significantly optimizes the perception pipeline. Achieving 95% of peak classification accuracy with only approximately 38.75% of the original data volume effectively alleviates storage pressure and reduces computational cost, as evidenced by a 61.25% reduction in data storage and a 54.10% decrease in training time. Overall, TTSPD introduces a novel and practical data morphology that supports efficient, high-performance perception under vehicle-grade computational constraints, offering valuable insights and resources for the future development of intelligent transportation systems.
- Multi-parameter sensing,
- Tire sensor data,
- Traffic scene image data,
- Road surface classification

FullText(HTML)

References(45)

References

[1]	QIAN Hui, WANG Mingchen, ZHU Maotao, et al. A review of multi-sensor fusion in autonomous driving[J]. Sensors, 2025, 25(19): 6033. doi: 10.3390/s25196033.
[2]	党宏社, 肖利霞, 张选德. 不良光照场景下的交通标志识别算法[J]. 半导体光电, 2025, 46(1): 142–148. doi: 10.16818/j.issn1001-5868.20240924001. DANG Hongshe, XIAO Lixia, and ZHANG Xuande. Traffic sign recognition algorithm under adverse lighting conditions[J]. Semiconductor Optoelectronics, 2025, 46(1): 142–148. doi: 10.16818/j.issn1001-5868.20240924001.
[3]	YAO Shanliang, GUAN Runwei, HUANG Xiaoyu, et al. Radar-camera fusion for object detection and semantic segmentation in autonomous driving: A comprehensive review[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(1): 2094–2128. doi: 10.1109/TIV.2023.3307157.
[4]	李奕, 张明, 段文瑞, 等. 光学参数计量评估在道路交通场景中的应用及研究进展(特邀)[J]. 光子学报, 2025, 54(11): 1154304. doi: 10.3788/gzxb20255411.1154304. LI Yi, ZHANG Ming, DUAN Wenrui, et al. Application and research progress of optical parameter metrology and evaluation in traffic scenarios (invited)[J]. Acta Photonica Sinica, 2025, 54(11): 1154304. doi: 10.3788/gzxb20255411.1154304.
[5]	曲立国, 张鑫, 卢自宝, 等. 基于改进YOLOv5的交通标志识别方法[J]. 光电工程, 2024, 51(6): 240055. doi: 10.12086/oee.2024.240055. QU Liguo, ZHANG Xin, LU Zibao, et al. A traffic sign recognition method based on improved YOLOv5[J]. Opto-Electronic Engineering, 2024, 51(6): 240055. doi: 10.12086/oee.2024.240055.
[6]	DONG Zhekang, GU Shenyu, ZHOU Shiqi, et al. Periodic segmentation transformer-based internal short circuit detection method for battery packs[J]. IEEE Transactions on Transportation Electrification, 2025, 11(1): 3655–3666. doi: 10.1109/TTE.2024.3444453.
[7]	WANG Yan, YIN Guodong, HANG Peng, et al. Fundamental estimation for tire road friction coefficient: A model-based learning framework[J]. IEEE Transactions on Vehicular Technology, 2025, 74(1): 481–493. doi: 10.1109/TVT.2024.3464524.
[8]	GU Tianli, LI Bo, QUAN Zhenqiang, et al. A novel estimation method for tire-road friction coefficient using intelligent tire and tire dynamics[J]. Mechanical Systems and Signal Processing, 2025, 235: 112872. doi: 10.1016/j.ymssp.2025.112872.
[9]	TAO Siyou, JU Zhiyang, LI Liang, et al. Tire road friction coefficient estimation for individual wheel based on two robust PMI observers and a multilayer perceptron[J]. IEEE Transactions on Vehicular Technology, 2024, 73(9): 12530–12541. doi: 10.1109/TVT.2024.3390032.
[10]	JI Xiaoyue, HAN Yifeng, LAI C S, et al. ViP-HMNN: A visual pathway-inspired hybrid neural network incorporated with in-memory computing for object recognition[J]. Information Fusion, 2026, 130: 104086. doi: 10.1016/j.inffus.2025.104086.
[11]	KIM S, KIM Y J, LEE D, et al. Robust road surface classification using time series augmented intelligent tire sensor data and 1-D CNN[J]. IEEE Access, 2025, 13: 76508–76515. doi: 10.1109/ACCESS.2025.3565656.
[12]	HAN Zongzhi, LIU Weidong, GAO Zhenhai, et al. A method for real-time road surface identification of intelligent tire systems based on random convolutional kernel neural network[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(10): 6487–6501. doi: 10.1109/TIV.2024.3369951.
[13]	KARKARIA V, CHEN Jie, LUEY C, et al. A digital twin framework utilizing machine learning for robust predictive maintenance: Enhancing tire health monitoring[J]. Journal of Computing and Information Science in Engineering, 2025, 25(7): 071003. doi: 10.1115/1.4067270.
[14]	YANG Yiting, XIAO Yao, TAN Yingqi, et al. Multimodal sensor fusion for road surface identification considering vehicle dynamic characteristics[C]. Proceedings of 2025 IEEE Intelligent Vehicles Symposium (IV), Cluj-Napoca, Romania, 2025: 1825–1832. doi: 10.1109/IV64158.2025.11097345.
[15]	YOON Y, KIM H, LEE S K, et al. Tire–road friction estimation and classification based on a CNN using tire acoustical signals for autonomous driving vehicles[R]. SAE Technical Paper 2025-01-8761, 2025. doi: 10.4271/2025-01-8761.
[16]	DONG Zhekang, ZHU Liyan, ZHOU Shiqi, et al. FE-SpikeFormer: A camera-based facial expression recognition method for hospital health monitoring[J]. IEEE Journal of Biomedical and Health Informatics, 2025: 1–11. doi: 10.1109/JBHI.2025.3589267. (查阅网上资料,未找到对应的卷期页码信息,请确认).
[17]	杨宇翔, 曹旗, 高明煜, 等. 基于多阶段多尺度彩色图像引导的道路场景深度图像补全[J]. 电子与信息学报, 2022, 44(11): 3951–3959. doi: 10.11999/JEIT210967. YANG Yuxiang, CAO Qi, GAO Mingyu, et al. Multi-stage multi-scale color guided depth image completion for road scenes[J]. Journal of Electronics & Information Technology, 2022, 44(11): 3951–3959. doi: 10.11999/JEIT210967.
[18]	CAESAR H, BANKITI V, LANG A H, et al. nuScenes: A multimodal dataset for autonomous driving[C]. Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 11618–11628. doi: 10.1109/CVPR42600.2020.01164.
[19]	SUN Pei, KRETZSCHMAR H, DOTIWALLA X, et al. Scalability in perception for autonomous driving: Waymo open dataset[C]. Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 2443–2451. doi: 10.1109/CVPR42600.2020.00252.
[20]	DE S. TOLEDO R, DE OLIVEIRA C S, ANDALÓ F, et al. FCDD: A high-resolution unstructured environment dataset with multiple sand roads[J]. IEEE Access, 2025, 13: 191531–191542. doi: 10.1109/ACCESS.2025.3630348.
[21]	ZHAO Tong, HE Junxiang, LV Jingcheng, et al. A comprehensive implementation of road surface classification for vehicle driving assistance: Dataset, models, and deployment[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(8): 8361–8370. doi: 10.1109/TITS.2023.3264588.
[22]	ZHAO Tong, GUO Peilin, and WEI Yintao. Road friction estimation based on vision for safe autonomous driving[J]. Mechanical Systems and Signal Processing, 2024, 208: 111019. doi: 10.1016/j.ymssp.2023.111019.
[23]	ZHAO Tong. RSCD: Road surface classification dataset with detailed annotations for driving assistance[DB/OL]. IEEE Dataport. https://doi.org/10.21227/446p-xr65, 2022.
[24]	ARYA D, MAEDA H, GHOSH S K, et al. Deep learning-based road damage detection and classification for multiple countries[J]. Automation in Construction, 2021, 132: 103935. doi: 10.1016/j.autcon.2021.103935.
[25]	ARYA D, MAEDA H, GHOSH S K, et al. Global road damage detection: State-of-the-art solutions[C]. Proceedings of 2020 IEEE International Conference on Big Data, Atlanta, USA, 2020: 5533–5542. doi: 10.1109/BigData50022.2020.9377790.
[26]	GÄHLERT N, JOURDAN N, CORDTS M, et al. Cityscapes 3D: Dataset and benchmark for 9 DoF vehicle detection[C]. Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, USA, 2021: 1886–1895. doi: 10.1109/WACV48630.2021.00193. (查阅网上资料,未找到本条文献信息且doi打开与本条文献内容不相符,请确认).
[27]	FENG Di, HAASE-SCHUTZ C, ROSENBAUM L, et al. Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(3): 1341–1360. doi: 10.1109/TITS.2020.2972974.
[28]	YEONG D J, VELASCO-HERNANDEZ G, BARRY J, et al. Sensor and sensor fusion technology in autonomous vehicles: A review[J]. Sensors, 2021, 21(6): 2140. doi: 10.3390/s21062140.
[29]	KUUTTI S, FALLAH S, KATSAROS K, et al. A survey of the state-of-the-art localization techniques and their potentials for autonomous vehicle applications[J]. IEEE Internet of Things Journal, 2018, 5(2): 829–846. doi: 10.1109/JIOT.2018.2812300.
[30]	HUANG Jiye, CHEN Xinshi, JIN Qingsong, et al. A fusion estimation method for tire-road friction coefficient based on weather and road images[J]. Lubricants, 2025, 13(10): 459. doi: 10.3390/lubricants13100459.
[31]	QIU Zhimin, SHAO Jinju, GUO Dong, et al. A multi-feature fusion approach for road surface recognition leveraging millimeter-wave radar[J]. Sensors, 2025, 25(12): 3802. doi: 10.3390/s25123802.
[32]	LIU Shaoshan, LIU Liangkai, TANG Jie, et al. Edge computing for autonomous driving: Opportunities and challenges[J]. Proceedings of the IEEE, 2019, 107(8): 1697–1716. doi: 10.1109/JPROC.2019.2915983.
[33]	BUDA M, MAKI A, and MAZUROWSKI M A. A systematic study of the class imbalance problem in convolutional neural networks[J]. Neural Networks, 2018, 106: 249–259. doi: 10.1016/j.neunet.2018.07.011.
[34]	KANG Bingyi, XIE Saining, ROHRBACH M, et al. Decoupling representation and classifier for long-tailed recognition[C]. Proceedings of the 8th International Conference on Learning Representations, Addis Ababa, Ethiopia, 2020.
[35]	DÓZSA T, JURDANA V, ŠEGOTA S B, et al. Road type classification using time-frequency representations of tire sensor signals[J]. IEEE Access, 2024, 12: 53361–53372. doi: 10.1109/ACCESS.2024.3382931.
[36]	WU Ti, ZHANG Xiaolong, WANG Dong, et al. Comparative study and real-world validation of vertical load estimation techniques for intelligent tire systems[J]. Sensors, 2025, 25(7): 2100. doi: 10.3390/s25072100.
[37]	THARWAT A. Classification assessment methods[J]. Applied Computing and Informatics, 2021, 17(1): 168–192. doi: 10.1016/j.aci.2018.08.003.
[38]	任俊宇, 俞宁宁, 周成伟, 等. DroneRFb-DIR: 用于非合作无人机个体识别的射频信号数据集[J]. 电子与信息学报, 2025, 47(3): 573–581. doi: 10.11999/JEIT240804. REN Junyu, YU Ningning, ZHOU Chengwei, et al. DroneRFb-DIR: An RF signal dataset for non-cooperative drone individual identification[J]. Journal of Electronics & Information Technology, 2025, 47(3): 573–581. doi: 10.11999/JEIT240804.
[39]	俞宁宁, 毛盛健, 周成伟, 等. DroneRFa: 用于侦测低空无人机的大规模无人机射频信号数据集[J]. 电子与信息学报, 2024, 46(4): 1147–1156. doi: 10.11999/JEIT230570. YU Ningning, MAO Shengjian, ZHOU Chengwei, et al. DroneRFa: A large-scale dataset of drone radio frequency signals for detecting low-altitude drones[J]. Journal of Electronics & Information Technology, 2024, 46(4): 1147–1156. doi: 10.11999/JEIT230570.
[40]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. doi: 10.1109/CVPR.2016.90.
[41]	TAN Mingxing and LE Q V. EfficientNet: Rethinking model scaling for convolutional neural networks[C]. Proceedings of the 36th International Conference on Machine Learning, Long Beach, USA, 2019: 6105–6114.
[42]	HOWARD A, SANDLER M, CHEN Bo, et al. Searching for MobileNetV3[C]. Proceedings of 2019 IEEE/CVF International Conference on Computer Vision, Seoul, South Korea, 2019: 1314–1324. doi: 10.1109/ICCV.2019.00140.
[43]	MA Ningning, ZHANG Xiangyu, ZHENG Haitao, et al. ShuffleNet V2: Practical guidelines for efficient CNN architecture design[C]. Proceedings of the 15th European Conference on Computer Vision, Munich, Germany, 2018: 122–138. doi: 10.1007/978-3-030-01264-9_8.
[44]	PAN S J and YANG Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345–1359. doi: 10.1109/TKDE.2009.191.
[45]	DENG Jia, DONG Wei, SOCHER R, et al. ImageNet: A large-scale hierarchical image database[C]. Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, 2009: 248–255. doi: 10.1109/CVPR.2009.5206848.