Object Detection and Tracking of Unmanned Surface Vehicles Based on Spatial-temporal Information Fusion

Zhiguo ZHOU; Zhao JING; Qiuling WANG; Chong QU

doi:10.11999/JEIT200223

Volume 43 Issue 6

Jun. 2021

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2021 > 43(6): 1698-1705

Zhiguo ZHOU, Zhao JING, Qiuling WANG, Chong QU. Object Detection and Tracking of Unmanned Surface Vehicles Based on Spatial-temporal Information Fusion[J]. Journal of Electronics & Information Technology, 2021, 43(6): 1698-1705. doi: 10.11999/JEIT200223

Citation:

Zhiguo ZHOU, Zhao JING, Qiuling WANG, Chong QU. Object Detection and Tracking of Unmanned Surface Vehicles Based on Spatial-temporal Information Fusion[J]. Journal of Electronics & Information Technology, 2021, 43(6): 1698-1705. doi: 10.11999/JEIT200223

Citation:

PDF( 3837 KB)

Object Detection and Tracking of Unmanned Surface Vehicles Based on Spatial-temporal Information Fusion

doi: 10.11999/JEIT200223

Zhiguo ZHOU^{1
,
,},
Zhao JING¹,
Qiuling WANG¹,
Chong QU^{1, 2}

1.
School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China
2.
Shanghai Marine Diesel Engine Research Institute, China State Shipbuilding Corporation Limited, Shanghai 201108, China

Received Date: 2020-03-31
Rev Recd Date: 2020-09-29

Available Online: 2020-09-30

Publish Date: 2021-06-18

Abstract

Abstract

Object detection and tracking is essential in the navigation, obstacle avoidance and other tasks of Unmanned Surface Vehicles (USV). However, the environment on the water is complex, and there are many problems such as object scale variation, occlusion, illumination variation and camera shaking, etc. This paper proposes the visual object detection and tracking of USV based on spatial-temporal information fusion. Deep learning detection in space is used to extract single-frame depth semantic features and correlation filter tracking in time is used to calculate the correlation of oriented gradient feature between frames. Temporal and spatial information through feature comparison are combined to achieve continuous and stable object detection and tracking with strong robustness at real-time. The experiments results demonstrate that the average detection and tracking accuracy is 0.83 with the average running speed of 15 fps, which illustrates the accuracy is improved and the speed is high.
- Unmanned Surface Vehicle (USV),
- Surface vehicle object,
- Detection and tracking,
- Spatial-temporal information Fusion

FullText(HTML)

References(21)

References

[1]	李寰宇, 毕笃彦, 杨源, 等. 基于深度特征表达与学习的视觉跟踪算法研究[J]. 电子与信息学报, 2015, 37(9): 2033–2039. doi: 10.11999/JEIT150031 LI Huanyu, BI Duyan, YANG Yuan, et al. Research on visual tracking algorithm based on deep feature expression and learning[J]. Journal of Electronics &Information Technology, 2015, 37(9): 2033–2039. doi: 10.11999/JEIT150031
[2]	WANG Bo, SU Yumin, and WAN Lei. A sea-sky line detection method for unmanned surface vehicles based on gradient saliency[J]. Sensors, 2016, 16(4): 543. doi: 10.3390/s16040543
[3]	KRISTAN M, KENK V S, KOVAČIČ S, et al. Fast image-based obstacle detection from unmanned surface vehicles[J]. IEEE Transactions on Cybernetics, 2016, 46(3): 641–654. doi: 10.1109/TCYB.2015.2412251
[4]	REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[C]. International Conference on Neural Information Processing Systems, Istanbul, Turkey, 2015: 91–99.
[5]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 779–788. doi: 10.1109/Cvpr.2016.91.
[6]	LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: Single shot MultiBox detector[C]. 14th European Conference on Computer Vision, The Netherlands, 2016: 21–37. doi: 10.1007/978-3-319-46448-0_2.
[7]	QIN Yueming and ZHANG Xiuzhi. Robust obstacle detection for unmanned surface vehicles[C]. MIPPR 2017: Remote Sensing Image Processing, Geographic Information Systems, and Other Applications, Xiangyang, China, 2018: 10611E1–10611E6. doi: 10.1117/12.2285607.
[8]	YANG Jian, XIAO Yang, FANG Zhiwen, et al. An object detection and tracking system for unmanned surface vehicles[C]. Target and Background Signatures III, Warsaw, Poland, 2017: 104320R1–104320R8. doi: 10.1117/12.2278220.
[9]	COMANICIU D, RAMESH V, and MEER P. Real-time tracking of non-rigid objects using mean shift[C]. The 2000 IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head, USA, 2000: 142–149. doi: 10.1109/CVPR.2000.854761.
[10]	LI Yongmin. On incremental and robust subspace learning[J]. Pattern Recognition, 2004, 37(7): 1509–1518. doi: 10.1016/j.patcog.2003.11.010
[11]	XUE Mei and LING Haibin. Robust visual tracking using ℓ1 minimization[C]. The 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 2009: 1436–1443. doi: 10.1109/ICCV.2009.5459292.
[12]	OZA N C. Online bagging and boosting[C]. The 2005 IEEE International Conference on Systems, Man and Cybernetics, Waikoloa, USA, 2005: 2340–2345. doi: 10.1109/ICSMC.2005.1571498.
[13]	HARE S, SAFFARI A, and TORR P H S. Struck: Structured output tracking with kernels[C]. The 2011 International Conference on Computer Vision, Barcelona, Spain, 2011: 263–270. doi: 10.1109/ICCV.2011.6126251.
[14]	BREIMAN L. Random forests[J]. Machine Learning, 2001, 45(1): 5–32. doi: 10.1023/A:1010933404324
[15]	侯志强, 王帅, 廖秀峰, 等. 基于样本质量估计的空间正则化自适应相关滤波视觉跟踪[J]. 电子与信息学报, 2019, 41(8): 1983–1991. doi: 10.11999/JEIT180921 HOU Zhiqiang, WANG Shuai, LIAO Xiufeng, et al. Adaptive regularized correlation filters for visual tracking based on sample quality estimation[J]. Journal of Electronics &Information Technology, 2019, 41(8): 1983–1991. doi: 10.11999/JEIT180921
[16]	BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, USA, 2010: 2544–2550. doi: 10.1109/CVPR.2010.5539960.
[17]	HENRIQUES J F, CASEIRO R, MARTINS P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]. Proceedings of the 12th European Conference on Computer Vision, Florence, Italy, 2012: 702–715. doi: 10.1007/978-3-642-33765-9_50.
[18]	LI Yang and ZHU Jianke. A scale adaptive kernel correlation filter tracker with feature integration[C]. 2014 European Conference on Computer Vision (ECCV) Workshops, Zurich, 2014: 254–265. doi: 10.1007/978-3-319-16181-5_18.
[19]	DANELLJAN M, HÄGER G, KHAN F S, et al. Accurate scale estimation for robust visual tracking[C]. British Machine Vision Conference, Nottingham, UK, 2014: 65.1–65.11. doi: 10.5244/C.28.65.
[20]	王鹏, 孙梦宇, 王海燕, 等. 一种目标响应自适应的通道可靠性跟踪算法[J]. 电子与信息学报, 2020, 42(8): 1950–1958. doi: 10.11999/JEIT190569 WANG Peng, SUN Mengyu, WANG Haiyan, et al. An object tracking algorithm with channel reliability and target response adaptation[J]. Journal of Electronics &Information Technology, 2020, 42(8): 1950–1958. doi: 10.11999/JEIT190569
[21]	HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583–596. doi: 10.1109/tpami.2014.2345390