Metro Pedestrian Detection Algorithm Based on Multi-scale Weighted Feature Fusion Network
-
摘要: 随着地铁乘客的大量增加,实时准确地监测地铁站内客流量对于保证乘客安全具有重要意义。针对地铁场景复杂、行人目标小等特点,该文提出了多尺度加权特征融合(MWF)网络,实现地铁客流量的精准实时监测。在数据预处理阶段,该文提出过采样目标增强算法,对小目标占比不足的图片进行拼接处理,增加小目标在训练时的迭代频率。其次,在单镜头多核检测器(SSD)网络基础上添加了基于VGG16网络的特征提取层,将不同尺度的特征层以不同方式进行加权融合,并选出最优的特征融合方式。最终,结合小目标过采样增强算法,得到多尺度加权特征融合模型。实验证明,该方法与SSD网络相比,在保证实时性的同时,检测精度提升了5.82%。Abstract: With the large increase of passengers in metro stations, precise and real-time monitoring of passenger flow in subway stations is of great significance for ensuring passenger safety. Based on the features of complicated subway scenes and small pedestrian targets, a Multi-scale Weighted Feature (MWF) fusion network to achieve accurate real-time monitoring of subway passengers is proposed. In the data preprocessing stage, an oversampling target enhancement algorithm is proposed to stitch the pictures with an insufficient proportion of small targets to increase the iteration frequency of small targets during training. Secondly, feature extraction layers based on the VGG16 network are added to the Single Shot multibox Detector (SSD) network. The feature layers of different scales are weighted and fused in different ways, and the optimal feature fusion method is selected. Finally, combined with the small target oversampling enhancement algorithm, a multi-scale weighted feature fusion model is obtained. Experiments show that the detection accuracy of this method has improved by 5.82 percent compared with the SSD network and doesn’t reduce the speed of detection.
-
Key words:
- Target detection /
- Small target /
- Deep network /
- Weighted feature fusion
-
表 1 特征层融合精度结果比较
方法 预训练 融合层 mAP SSD300 × None 64.89 SSD300 √ None 83.90 MWFSSD300 × Conv4+Conv7+ Conv8 82.79 MWFSSD300 √ Conv4+Conv7+ Conv8 85.12 MWFSSD300 √ Conv3+Conv7 83.67 MWFSSD300 √ Conv7+ Conv8 82.10 MWFSSD300 √ Conv3+Conv4+Conv7+ Conv8 86.48 MWFSSD300 √ Conv4+Conv7+ Conv8 85.22 表 2 不同权重分配的检测结果
权重分配方式 w1 w2 w3 w4 mAP 0 0 0 0 0 86.48 1 0.4 0.3 0.2 0.1 86.39 2 0.5 0.3 0.1 0.1 87.92 3 0.2 0.2 0.3 0.3 86.44 表 3 MWFSSD与主流检测方法检测结果对比
网络 mAP(%) fps(frame/s) MWFSSD 89.72 32 SSD 83.90 43 Faster-RCNN 88.86 20 YoloV3 86.50 38 -
[1] GIRSHICK R. Fast R-CNN[C]. 2015 IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 1440–1448. doi: 10.1109/ICCV.2015.169. [2] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149. doi: 10.1109/TPAMI.2016.2577031 [3] 王殿伟, 何衍辉, 李大湘, 等. 改进的YOLOv3红外视频图像行人检测算法[J]. 西安邮电大学学报, 2018, 23(4): 48–52, 67. doi: 10.13682/j.issn.2095-6533.2018.04.008WANG Dianwei, HE Yanhui, LI Daxiang, et al. An improved infrared video image pedestrian detection algorithm[J]. Journal of Xi’an University of Posts and Telecommunications, 2018, 23(4): 48–52, 67. doi: 10.13682/j.issn.2095-6533.2018.04.008 [4] LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: Single shot MultiBox detector[J]. Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands, 2016: 21–37. doi: 10.1007/978-3-319-46448-0_2 [5] LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 936–944. doi: 10.1109/CVPR.2017.106. [6] 吕俊奇, 邱卫根, 张立臣, 等. 多层卷积特征融合的行人检测[J]. 计算机工程与设计, 2018, 39(11): 3481–3485. doi: 10.16208/j.issn1000-7024.2018.11.032LÜ Junqi, QIU Weigen, ZHANG Lichen, et al. Multi-scale convolutional feature fusion for pedestrian detection[J]. Computer Engineering and Design, 2018, 39(11): 3481–3485. doi: 10.16208/j.issn1000-7024.2018.11.032 [7] 张文明, 姚振飞, 高雅昆, 等. 一种平衡准确性以及高效性的显著性目标检测深度卷积网络模型[J]. 电子与信息学报, 2020, 42(5): 1201–1208. doi: 10.11999/JEIT190229ZHANG Wenming, YAO Zhenfei, GAO Yakun, et al. A deep convolutional network for saliency object detection with balanced accuracy and high efficiency[J]. Journal of Electronics &Information Technology, 2020, 42(5): 1201–1208. doi: 10.11999/JEIT190229 [8] 刘晴, 唐林波, 赵保军, 等. 基于自适应多特征融合的均值迁移红外目标跟踪[J]. 电子与信息学报, 2012, 34(5): 1137–1141. doi: 10.3724/SP.J.1146.2011.01077LIU Qing, TANG Linbo, ZHAO Baojun, et al. Infrared target tracking based on adaptive multiple features fusion and mean shift[J]. Journal of Electronics &Information Technology, 2012, 34(5): 1137–1141. doi: 10.3724/SP.J.1146.2011.01077 [9] 颜伟, 耿路, 周雷, 等. 基于海情和三次样条插值算法的舰船雷达散射截面优化分析方法[J]. 电子与信息学报, 2018, 40(3): 579–586. doi: 10.11999/JEIT170562YAN Wei, GENG Lu, ZHOU Lei, et al. Optimization analysis method on ship RCS based on sea conditions and cubic spline interpolation algorithm[J]. Journal of Electronics &Information Technology, 2018, 40(3): 579–586. doi: 10.11999/JEIT170562 [10] 邓苗, 张基宏, 柳伟, 等. 基于全变分的权值优化的多尺度变换图像融合[J]. 电子与信息学报, 2013, 35(7): 1657–1663. doi: 10.3724/SP.J.1146.2012.01183DENG Miao, ZHANG Jihong, LIU Wei, et al. A total variation-based lowpass weight function optimization in multiscale image fusion[J]. Journal of Electronics &Information Technology, 2013, 35(7): 1657–1663. doi: 10.3724/SP.J.1146.2012.01183 [11] 李秋华, 李吉成, 沈振康. 基于多尺度特征融合的红外图像小目标检测[J]. 系统工程与电子技术, 2005, 27(9): 1557–1560. doi: 10.3321/j.issn:1001-506X.2005.09.018LI Qiuhua, LI Jicheng, and SHEN Zhenkang. IR image small target detection based on multi-scale feature fusion[J]. Systems Engineering and Electronics, 2005, 27(9): 1557–1560. doi: 10.3321/j.issn:1001-506X.2005.09.018 [12] 姜文涛, 张驰, 张晟翀, 等. 多尺度特征图融合的目标检测[J]. 中国图象图形学报, 2019, 24(11): 1918–1931. doi: 10.11834/jig.190021JIANG Wentao, ZHANG Chi, ZHANG Shengchong, et al. Multiscale feature map fusion algorithm for target detection[J]. Journal of Image and Graphics, 2019, 24(11): 1918–1931. doi: 10.11834/jig.190021 [13] 王瑶, 王正勇, 何小海, 等. 基于多尺度训练库与多特征融合的人脸识别[J]. 电视技术, 2015, 39(1): 121–126. doi: 10.16280/j.videoe.2015.01.031WANG Yao, WANG Zhengyong, HE Xiaohai, et al. Face recognition by features fusion based on multiscale training set[J]. Video Engineering, 2015, 39(1): 121–126. doi: 10.16280/j.videoe.2015.01.031 [14] 余春艳, 徐小丹, 钟诗俊. 面向显著性目标检测的SSD改进模型[J]. 电子与信息学报, 2018, 40(11): 2554–2561. doi: 10.11999/JEIT180118YU Chunyan, XU Xiaodan, and ZHONG Shijun. An improved SSD model for saliency object detection[J]. Journal of Electronics &Information Technology, 2018, 40(11): 2554–2561. doi: 10.11999/JEIT180118 [15] 孙彦景, 石韫开, 云霄, 等. 基于多层卷积特征的自适应决策融合目标跟踪算法[J]. 电子与信息学报, 2019, 41(10): 2464–2470. doi: 10.11999/JEIT180971SUN Yanjing, SHI Yunkai, YUN Xiao, et al. Adaptive strategy fusion target tracking based on multi-layer convolutional features[J]. Journal of Electronics &Information Technology, 2019, 41(10): 2464–2470. doi: 10.11999/JEIT180971 [16] 张思宇, 张轶. 基于多尺度特征融合的小目标行人检测[J]. 计算机工程与科学, 2019, 41(9): 1627–1634. doi: 10.3969/j.issn.1007-130X.2019.09.014ZHANG Siyu and ZHANG Yi. Small target pedestrian detection based on multi-scale feature fusion[J]. Computer Engineering and Science, 2019, 41(9): 1627–1634. doi: 10.3969/j.issn.1007-130X.2019.09.014 [17] DOLLAR P, WOJEK C, SCHIELE B, et al. Pedestrian detection: An evaluation of the state of the art[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(4): 743–761. doi: 10.1109/TPAMI.2011.155 [18] 汪荣贵, 韩梦雅, 杨娟, 等. 多级注意力特征网络的小样本学习[J]. 电子与信息学报, 2020, 42(3): 772–778. doi: 10.11999/JEIT190242WANG Ronggui, HAN Mengya, YANG Juan, et al. Multi-level attention feature network for few-shot learning[J]. Journal of Electronics &Information Technology, 2020, 42(3): 772–778. doi: 10.11999/JEIT190242 [19] 代科学, 李国辉, 涂丹, 等. 监控视频运动目标检测减背景技术的研究现状和展望[J]. 中国图象图形学报, 2007, 11(7): 919–927. doi: 10.3969/j.issn.1006-8961.2006.07.002DAI Kexue, LI Guohui, TU Dan, et al. Prospects and current studies on background subtraction techniques for moving objects detection from surveillance video[J]. Journal of Image and Graphics, 2007, 11(7): 919–927. doi: 10.3969/j.issn.1006-8961.2006.07.002 [20] 陈勇, 刘曦, 刘焕淋. 基于特征通道和空间联合注意机制的遮挡行人检测方法[J]. 电子与信息学报, 2020, 42(6): 1486–1493. doi: 10.11999/JEIT190606CHEN Yong, LIU Xi, and LIU Huanlin. Occluded pedestrian detection based on joint attention mechanism of channel-wise and spatial information[J]. Journal of Electronics &Information Technology, 2020, 42(6): 1486–1493. doi: 10.11999/JEIT190606 [21] 贺丰收, 何友, 刘准钆, 等. 卷积神经网络在雷达自动目标识别中的研究进展[J]. 电子与信息学报, 2020, 42(1): 119–131. doi: 10.11999/JEIT180899HE Fengshou, HE You, LIU Zhunga, et al. Research and development on applications of convolutional neural networks of radar automatic target recognition[J]. Journal of Electronics &Information Technology, 2020, 42(1): 119–131. doi: 10.11999/JEIT180899