TTSPD: A Multimodal Traffic Scene Perception Dataset Integrating Tire Data

YING Zongchen; GUI Lin; YANG Jiahan; ZHANG Fangwei; WANG Junfan; DONG Zhekang

doi:10.11999/JEIT260022

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

YING Zongchen, GUI Lin, YANG Jiahan, ZHANG Fangwei, WANG Junfan, DONG Zhekang. TTSPD: A Multimodal Traffic Scene Perception Dataset Integrating Tire Data[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260022

Citation:

YING Zongchen, GUI Lin, YANG Jiahan, ZHANG Fangwei, WANG Junfan, DONG Zhekang. TTSPD: A Multimodal Traffic Scene Perception Dataset Integrating Tire Data[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260022

Citation:

PDF( 3706 KB)

TTSPD: A Multimodal Traffic Scene Perception Dataset Integrating Tire Data

doi: 10.11999/JEIT260022 cstr: 32379.14.JEIT260022

YING Zongchen^{1, 2},
GUI Lin^{1, 2},
YANG Jiahan^{1, 2},
ZHANG Fangwei³,
WANG Junfan^{1, 2},
DONG Zhekang^{1, 2, 4
,
,}

1.
School of Electronic Information, Hangzhou Dianzi University, Hangzhou 310018, China
2.
Zhejiang Key Laboratory of Intelligent Vehicle Electronics Research, Hangzhou 310018, China
3.
BH SENS (Shanghai) Electronics Co., Ltd., Shanghai 210619, China
4.
College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China

Funds: Yangtze River Delta Science and Technology Innovation Program (YDZX20233100004028), Zhejiang Provincial Natural Science Foundation of China (LZYQ25F020005)

Received Date: 2026-01-07
Accepted Date: 2026-02-05
Rev Recd Date: 2026-02-03

Available Online: 2026-02-27

Abstract

Abstract

Objective With the rapid development of Intelligent Transportation Systems (ITS) and autonomous driving technologies, accurate traffic environment perception is a fundamental prerequisite for vehicle safety and decision making. Current perception frameworks primarily rely on high-resolution cameras and LiDAR sensors. Although these sensors provide rich information, they create severe challenges across the Perception-Storage-Calculation pipeline. High acquisition costs limit large-scale deployment. In addition, the massive data volume produced by high-dimensional sensors places heavy pressure on onboard storage and computational resources, often exceeding the power and thermal budgets of vehicle-grade edge platforms. These constraints motivate the exploration of alternative sensing paradigms that are cost-effective, compact, and computationally efficient while maintaining reliable perception accuracy. In response, the present study shifts the perception perspective from conventional external sensors to the tire-road contact interface, where abundant physical interaction information naturally exists. The objective is to construct a novel multimodal dataset, termed the Tire-integrated Traffic Scene Perception Dataset (TTSPD), which combines internal tire dynamics with external visual observations. This dataset is used to examine whether low-dimensional tire sensing data can complement or partially substitute high-dimensional visual data for accurate road surface classification. The study also aims to establish a new data morphology that balances perception performance and system efficiency for future intelligent vehicles. Methods To construct a high-quality and practically usable multimodal dataset, an integrated hardware-software acquisition framework is developed. From a hardware perspective, a specialized sensing system is designed by coupling tire-mounted multi-parameter sensors with a vehicle-mounted camera. To ensure reliable operation under the harsh mechanical conditions of a rotating tire, sensing nodes are encapsulated using a rubber-based composite material that provides mechanical protection and long-term stability. Wireless transmission is implemented using Bluetooth Low Energy (BLE) 5.0 with an adaptive frequency-hopping mechanism, enabling low-power and reliable communication during high-speed rotation. During data acquisition, the system synchronously collects six types of internal tire signals, including radial acceleration, tire temperature, and tire pressure, producing approximately 1.8 million sampling points. In parallel, a dashboard-mounted camera records high-resolution traffic scene images totaling 309 GB across four representative road surface conditions. To address the heterogeneity between high-frequency one-dimensional tire signals and two-dimensional visual data, a timestamp-based association strategy is adopted to achieve scene-level temporal alignment rather than strict frame-by-frame correspondence. Sensor sequences and image segments are grouped according to shared temporal windows and driving scenarios. This approach ensures semantic and temporal consistency at the scene level. The alignment strategy reflects practical deployment conditions and forms the basis of the final TTSPD dataset for multimodal fusion research. Results and Discussions The effectiveness of the proposed TTSPD is evaluated through comprehensive road surface classification experiments using mainstream deep learning models. Initial experiments based solely on visual data demonstrate strong baseline performance, with classification accuracies ranging from 87.25% to 93.75% (Table 7). These results confirm the quality and diversity of the visual modality in the dataset. The primary contribution of this study is the quantification of efficiency gains enabled by tire-based sensing. Comparative experiments progressively reduce the amount of visual data while integrating low-dimensional tire signals, particularly radial acceleration (Table 9). The results show that the multimodal model achieves approximately 95% of the full-data baseline accuracy while using only about 38.75% of the original data volume. This reduction in data dependency produces significant system-level benefits. Storage requirements decrease by approximately 61.25%, and overall model training time decreases by about 54.10% (Fig. 8). These findings indicate that tire dynamics encode high-value physical features related to road texture and surface conditions that complement visual cues. The proposed dataset therefore supports the development of lighter perception pipelines without reducing recognition performance. Conclusions This study addresses the long-standing Perception-Storage-Calculation bottleneck in vision-dominated autonomous driving systems by proposing the TTSPD. Multi-parameter sensors are embedded within tires using rubber-based encapsulation, and stable wireless communication is achieved through BLE 5.0. A robust tire-camera data acquisition system is therefore established. The resulting dataset covers four common and safety-critical road surface types: cement, asphalt, damaged, and water-covered roads. It provides a comprehensive foundation for multimodal perception research. Experimental results show that combining low-dimensional tire sensing data with visual information significantly improves perception efficiency. Approximately 95% of peak classification accuracy is achieved using only about 38.75% of the original data volume. This result effectively reduces storage pressure and computational cost, reflected in a 61.25% reduction in data storage and a 54.10% reduction in training time. The TTSPD dataset therefore proposes a practical data morphology that supports efficient and high-performance perception under vehicle-grade computational constraints. It also provides valuable resources for the future development of ITS.
- Multi-parameter sensing,
- Tire sensor data,
- Traffic scene image data,
- Road surface classification

FullText(HTML)

References(45)

References

[1]	QIAN Hui, WANG Mingchen, ZHU Maotao, et al. A review of multi-sensor fusion in autonomous driving[J]. Sensors, 2025, 25(19): 6033. doi: 10.3390/s25196033.
[2]	党宏社, 肖利霞, 张选德. 不良光照场景下的交通标志识别算法[J]. 半导体光电, 2025, 46(1): 142–148. doi: 10.16818/j.issn1001-5868.20240924001. DANG Hongshe, XIAO Lixia, and ZHANG Xuande. Traffic sign recognition algorithm under adverse lighting conditions[J]. Semiconductor Optoelectronics, 2025, 46(1): 142–148. doi: 10.16818/j.issn1001-5868.20240924001.
[3]	YAO Shanliang, GUAN Runwei, HUANG Xiaoyu, et al. Radar-camera fusion for object detection and semantic segmentation in autonomous driving: A comprehensive review[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(1): 2094–2128. doi: 10.1109/TIV.2023.3307157.
[4]	李奕, 张明, 段文瑞, 等. 光学参数计量评估在道路交通场景中的应用及研究进展(特邀)[J]. 光子学报, 2025, 54(11): 1154304. doi: 10.3788/gzxb20255411.1154304. LI Yi, ZHANG Ming, DUAN Wenrui, et al. Application and research progress of optical parameter metrology and evaluation in traffic scenarios (invited)[J]. Acta Photonica Sinica, 2025, 54(11): 1154304. doi: 10.3788/gzxb20255411.1154304.
[5]	曲立国, 张鑫, 卢自宝, 等. 基于改进YOLOv5的交通标志识别方法[J]. 光电工程, 2024, 51(6): 240055. doi: 10.12086/oee.2024.240055. QU Liguo, ZHANG Xin, LU Zibao, et al. A traffic sign recognition method based on improved YOLOv5[J]. Opto-Electronic Engineering, 2024, 51(6): 240055. doi: 10.12086/oee.2024.240055.
[6]	DONG Zhekang, GU Shenyu, ZHOU Shiqi, et al. Periodic segmentation transformer-based internal short circuit detection method for battery packs[J]. IEEE Transactions on Transportation Electrification, 2025, 11(1): 3655–3666. doi: 10.1109/TTE.2024.3444453.
[7]	WANG Yan, YIN Guodong, HANG Peng, et al. Fundamental estimation for tire road friction coefficient: A model-based learning framework[J]. IEEE Transactions on Vehicular Technology, 2025, 74(1): 481–493. doi: 10.1109/TVT.2024.3464524.
[8]	GU Tianli, LI Bo, QUAN Zhenqiang, et al. A novel estimation method for tire-road friction coefficient using intelligent tire and tire dynamics[J]. Mechanical Systems and Signal Processing, 2025, 235: 112872. doi: 10.1016/j.ymssp.2025.112872.
[9]	TAO Siyou, JU Zhiyang, LI Liang, et al. Tire road friction coefficient estimation for individual wheel based on two robust PMI observers and a multilayer perceptron[J]. IEEE Transactions on Vehicular Technology, 2024, 73(9): 12530–12541. doi: 10.1109/TVT.2024.3390032.
[10]	JI Xiaoyue, HAN Yifeng, LAI C S, et al. ViP-HMNN: A visual pathway-inspired hybrid neural network incorporated with in-memory computing for object recognition[J]. Information Fusion, 2026, 130: 104086. doi: 10.1016/j.inffus.2025.104086.
[11]	KIM S, KIM Y J, LEE D, et al. Robust road surface classification using time series augmented intelligent tire sensor data and 1-D CNN[J]. IEEE Access, 2025, 13: 76508–76515. doi: 10.1109/ACCESS.2025.3565656.
[12]	HAN Zongzhi, LIU Weidong, GAO Zhenhai, et al. A method for real-time road surface identification of intelligent tire systems based on random convolutional kernel neural network[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(10): 6487–6501. doi: 10.1109/TIV.2024.3369951.
[13]	KARKARIA V, CHEN Jie, LUEY C, et al. A digital twin framework utilizing machine learning for robust predictive maintenance: Enhancing tire health monitoring[J]. Journal of Computing and Information Science in Engineering, 2025, 25(7): 071003. doi: 10.1115/1.4067270.
[14]	YANG Yiting, XIAO Yao, TAN Yingqi, et al. Multimodal sensor fusion for road surface identification considering vehicle dynamic characteristics[C]. 2025 IEEE Intelligent Vehicles Symposium (IV), Cluj-Napoca, Romania, 2025: 1825–1832. doi: 10.1109/IV64158.2025.11097345.
[15]	YOON Y, KIM H, LEE S K, et al. Tire–road friction estimation and classification based on a CNN using tire acoustical signals for autonomous driving vehicles[R]. SAE Technical Paper 2025-01-8761, 2025. doi: 10.4271/2025-01-8761.
[16]	DONG Zhekang, ZHU Liyan, ZHOU Shiqi, et al. FE-SpikeFormer: A camera-based facial expression recognition method for hospital health monitoring[J]. IEEE Journal of Biomedical and Health Informatics, 2025: 1–11. doi: 10.1109/JBHI.2025.3589267.
[17]	杨宇翔, 曹旗, 高明煜, 等. 基于多阶段多尺度彩色图像引导的道路场景深度图像补全[J]. 电子与信息学报, 2022, 44(11): 3951–3959. doi: 10.11999/JEIT210967. YANG Yuxiang, CAO Qi, GAO Mingyu, et al. Multi-stage multi-scale color guided depth image completion for road scenes[J]. Journal of Electronics & Information Technology, 2022, 44(11): 3951–3959. doi: 10.11999/JEIT210967.
[18]	CAESAR H, BANKITI V, LANG A H, et al. nuScenes: A multimodal dataset for autonomous driving[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 11618–11628. doi: 10.1109/CVPR42600.2020.01164.
[19]	SUN Pei, KRETZSCHMAR H, DOTIWALLA X, et al. Scalability in perception for autonomous driving: Waymo open dataset[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 2443–2451. doi: 10.1109/CVPR42600.2020.00252.
[20]	DE S. TOLEDO R, DE OLIVEIRA C S, ANDALÓ F, et al. FCDD: A high-resolution unstructured environment dataset with multiple sand roads[J]. IEEE Access, 2025, 13: 191531–191542. doi: 10.1109/ACCESS.2025.3630348.
[21]	ZHAO Tong, HE Junxiang, LV Jingcheng, et al. A comprehensive implementation of road surface classification for vehicle driving assistance: Dataset, models, and deployment[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(8): 8361–8370. doi: 10.1109/TITS.2023.3264588.
[22]	ZHAO Tong, GUO Peilin, and WEI Yintao. Road friction estimation based on vision for safe autonomous driving[J]. Mechanical Systems and Signal Processing, 2024, 208: 111019. doi: 10.1016/j.ymssp.2023.111019.
[23]	ZHAO Tong. RSCD: Road surface classification dataset with detailed annotations for driving assistance[DB/OL]. IEEE Dataport. https://doi.org/10.21227/446p-xr65, 2022.
[24]	ARYA D, MAEDA H, GHOSH S K, et al. Deep learning-based road damage detection and classification for multiple countries[J]. Automation in Construction, 2021, 132: 103935. doi: 10.1016/j.autcon.2021.103935.
[25]	ARYA D, MAEDA H, GHOSH S K, et al. Global road damage detection: State-of-the-art solutions[C]. 2020 IEEE International Conference on Big Data, Atlanta, USA, 2020: 5533–5542. doi: 10.1109/BigData50022.2020.9377790.
[26]	GÄHLERT N, JOURDAN N, CORDTS M, et al. Cityscapes 3D: Dataset and benchmark for 9 DoF vehicle detection[C]. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, USA, 2021: 1886–1895. doi: 10.1109/WACV48630.2021.00193.
[27]	FENG Di, HAASE-SCHUTZ C, ROSENBAUM L, et al. Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(3): 1341–1360. doi: 10.1109/TITS.2020.2972974.
[28]	YEONG D J, VELASCO-HERNANDEZ G, BARRY J, et al. Sensor and sensor fusion technology in autonomous vehicles: A review[J]. Sensors, 2021, 21(6): 2140. doi: 10.3390/s21062140.
[29]	KUUTTI S, FALLAH S, KATSAROS K, et al. A survey of the state-of-the-art localization techniques and their potentials for autonomous vehicle applications[J]. IEEE Internet of Things Journal, 2018, 5(2): 829–846. doi: 10.1109/JIOT.2018.2812300.
[30]	HUANG Jiye, CHEN Xinshi, JIN Qingsong, et al. A fusion estimation method for tire-road friction coefficient based on weather and road images[J]. Lubricants, 2025, 13(10): 459. doi: 10.3390/lubricants13100459.
[31]	QIU Zhimin, SHAO Jinju, GUO Dong, et al. A multi-feature fusion approach for road surface recognition leveraging millimeter-wave radar[J]. Sensors, 2025, 25(12): 3802. doi: 10.3390/s25123802.
[32]	LIU Shaoshan, LIU Liangkai, TANG Jie, et al. Edge computing for autonomous driving: Opportunities and challenges[J]. Proceedings of the IEEE, 2019, 107(8): 1697–1716. doi: 10.1109/JPROC.2019.2915983.
[33]	BUDA M, MAKI A, and MAZUROWSKI M A. A systematic study of the class imbalance problem in convolutional neural networks[J]. Neural Networks, 2018, 106: 249–259. doi: 10.1016/j.neunet.2018.07.011.
[34]	KANG Bingyi, XIE Saining, ROHRBACH M, et al. Decoupling representation and classifier for long-tailed recognition[C]. The 8th International Conference on Learning Representations, Addis Ababa, Ethiopia, 2020.
[35]	DÓZSA T, JURDANA V, ŠEGOTA S B, et al. Road type classification using time-frequency representations of tire sensor signals[J]. IEEE Access, 2024, 12: 53361–53372. doi: 10.1109/ACCESS.2024.3382931.
[36]	WU Ti, ZHANG Xiaolong, WANG Dong, et al. Comparative study and real-world validation of vertical load estimation techniques for intelligent tire systems[J]. Sensors, 2025, 25(7): 2100. doi: 10.3390/s25072100.
[37]	THARWAT A. Classification assessment methods[J]. Applied Computing and Informatics, 2021, 17(1): 168–192. doi: 10.1016/j.aci.2018.08.003.
[38]	任俊宇, 俞宁宁, 周成伟, 等. DroneRFb-DIR: 用于非合作无人机个体识别的射频信号数据集[J]. 电子与信息学报, 2025, 47(3): 573–581. doi: 10.11999/JEIT240804. REN Junyu, YU Ningning, ZHOU Chengwei, et al. DroneRFb-DIR: An RF signal dataset for non-cooperative drone individual identification[J]. Journal of Electronics & Information Technology, 2025, 47(3): 573–581. doi: 10.11999/JEIT240804.
[39]	俞宁宁, 毛盛健, 周成伟, 等. DroneRFa: 用于侦测低空无人机的大规模无人机射频信号数据集[J]. 电子与信息学报, 2024, 46(4): 1147–1156. doi: 10.11999/JEIT230570. YU Ningning, MAO Shengjian, ZHOU Chengwei, et al. DroneRFa: A large-scale dataset of drone radio frequency signals for detecting low-altitude drones[J]. Journal of Electronics & Information Technology, 2024, 46(4): 1147–1156. doi: 10.11999/JEIT230570.
[40]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. doi: 10.1109/CVPR.2016.90.
[41]	TAN Mingxing and LE Q V. EfficientNet: Rethinking model scaling for convolutional neural networks[C]. The 36th International Conference on Machine Learning, Long Beach, USA, 2019: 6105–6114.
[42]	HOWARD A, SANDLER M, CHEN Bo, et al. Searching for MobileNetV3[C]. 2019 IEEE/CVF International Conference on Computer Vision, Seoul, South Korea, 2019: 1314–1324. doi: 10.1109/ICCV.2019.00140.
[43]	MA Ningning, ZHANG Xiangyu, ZHENG Haitao, et al. ShuffleNet V2: Practical guidelines for efficient CNN architecture design[C]. The 15th European Conference on Computer Vision, Munich, Germany, 2018: 122–138. doi: 10.1007/978-3-030-01264-9_8.
[44]	PAN S J and YANG Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345–1359. doi: 10.1109/TKDE.2009.191.
[45]	DENG Jia, DONG Wei, SOCHER R, et al. ImageNet: A large-scale hierarchical image database[C]. 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, 2009: 248–255. doi: 10.1109/CVPR.2009.5206848.