基于双支路的人群视频异常检测算法研究

蔡轶珩; 刘天浩; 刘嘉琦; 郭雅君; 胡绍斌

doi:10.11999/JEIT210341

基于双支路的人群视频异常检测算法研究

doi: 10.11999/JEIT210341

北京工业大学信息学部北京 100124

基金项目: 国家重点研发计划(2017YFC1703302)

详细信息

作者简介:
蔡轶珩：女，1974年生，博士，副教授，研究方向为图像与视频信号处理技术

刘天浩：男，1997年生，硕士生，研究方向为图像处理

刘嘉琦：男，1996年生，硕士生，研究方向为图像处理

郭雅君：女，1996年生，硕士生，研究方向为图像处理

胡绍斌：男，1995年生，硕士生，研究方向为图像处理

通讯作者:
蔡轶珩　caiyiheng@bjut.edu.cn

中图分类号: TN911.73; TP391
计量
- 文章访问数: 698
- HTML全文浏览量: 418
- PDF下载量: 126
- 被引次数: 0
出版历程
- 收稿日期: 2021-04-23
- 修回日期: 2021-09-15
- 网络出版日期: 2021-09-29
- 刊出日期: 2022-07-25

Research on Crowd Video Anomaly Detection Algorithm Based on Dual-branch

Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China

Funds: The National Key Research and Development Program of China(2017YFC1703302)

摘要

摘要: 该文对人群异常检测任务进行了研究，针对人群场景视频背景较冗余，易受光线与噪声影响以及实际部署等问题，提出了人群异常多尺度特征记忆网络(CaMsm-net)。为了从多角度判别异常的发生，并将各类信息更好地融合，该网络采用了一种双支路共享单元结构，将原始帧与经背景建模后的帧同时输入网络结构中，分别对两条支路进行预测，采用预测误差进行异常的判定，并且从实际应用角度出发，框架结合了深度可分离卷积、数据扩增等方法，保障了检测的准确性与部署的可行性。在公开的美国明尼苏达大学(UMN)人群数据集与实际监控下的火车站出站口数据集中的实验表明，曲线下面积(AUC)指标分别达到了99.2%与84.1%，平均检测准确率分别为95.9%与81.7%，证明了所提算法能够较好地检测到各类人群异常现象的发生，并具有更广的适用性。
- 人群异常检测 /
- 双支路共享 /
- 背景建模 /
- 时空融合
Abstract: This paper studies the task of crowd anomaly detection. Considering the problems of crowd scene video background being redundant, susceptible to light and noise, and actual deployment, a Crowd anomaly Multi-scale feature memory network (CaMsm-net) is proposed. In order to distinguish the occurrence of anomalies from multiple angles and integrate better various types of information, a dual-branch shared unit structure is adopted by the network, the original frame and the background modeled frame are simultaneously input into the network structure. To predict the two branches separately, the prediction error was used to determine the abnormality, and from the perspective of practical application, the depthwise separable convolutionand data augmentation methods are added to the framework to ensure the accuracy of detection and the feasibility of deployment. Experiments on the public University of MinNesota (UMN) population dataset and the actual monitoring of the train station exit dataset show that the Area Under Curve (AUC) indicators reach 99.2% and 84.1% respectively, and the average detection accuracy rates are 95.9% and 81.7%, which proves the proposed algorithm can better detect the occurrence of various crowd abnormalities and has wider applicability.
- Crowd anomaly detection /
- Dual-branch sharing /
- Background modeling /
- Space-time fusion

HTML全文

图 1 算法整体流程图

下载: 全尺寸图片幻灯片

图 2 高斯混合建模举例

下载: 全尺寸图片幻灯片

图 3 双支路共享单元示意图

下载: 全尺寸图片幻灯片

图 4 网络详细结构图

下载: 全尺寸图片幻灯片

图 5 采用的两种数据扩增方式

下载: 全尺寸图片幻灯片

图 6 数据集展示

下载: 全尺寸图片幻灯片

图 7 UMN数据集原始帧与预测差值图

下载: 全尺寸图片幻灯片

图 8 火车站数据集异常分数展示

下载: 全尺寸图片幻灯片

图 9 各数据集ROC曲线图

下载: 全尺寸图片幻灯片

表 1 各方法AUC指标比较(%)

方法名	UMN	火车站数据集
SFM^[8]	96.0	–
LSA^12]	98.0	–
Chaotic Invariants^[13]	99.0	–
GF^[9]	98.9
本文算法	99.2	84.1

下载: 导出CSV

表 2 UMN数据集异常事件检测准确度(%)

方法名	草坪的无规则四散	室内同向加速奔跑	广场无规则四散	总体值
SRC^[16]	90.52	78.48	92.70	84.70
SFM^[12]	84.41	82.35	90.83	85.09
EM^[16]	90.79	91.84	88.53	90.39
D-CM^[3]	92.45	91.37	91.86	91.89
C3D-GRNN	80.70	85.60	77.50	81.30
本文算法	97.10	95.20	97.60	95.90

下载: 导出CSV

表 3 火车站出站口异常事件检测准确度(%)

异常种类	ACC
人群大量聚集	79.8
异常走向	89.3
总体值	81.7

下载: 导出CSV

表 4 双支路共享单元模块消融实验结果(%)

	UMN	火车站数据集
只使用原始帧	98.9	82.7
使用原始帧与消除背景帧	99.2	84.1

下载: 导出CSV

表 5 深度可分离卷积消融实验结果

	AUC(%)	参数量(MB)	fps
使用普通卷积	84.6	173	23
使用深度可分离卷积	84.1	117	27

下载: 导出CSV

表 6 数据扩增消融实验结果(%)

	AUC
使用原始数据	83.2
使用数据扩增后的数据	84.1

下载: 导出CSV

参考文献(16)

[1]	LIU Wen, LUO Weixin, LIAN Dongze, et al. Future frame prediction for anomaly detection-A new baseline[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6536–6545.
[2]	LI Yuanyuan, CAI Yiheng, LIU Jiaqi, et al. Spatio-temporal unity networking for video anomaly detection[J]. IEEE Access, 2019, 7: 172425–172432. doi: 10.1109/ACCESS.2019.2954540
[3]	ZHOU Bolei, TANG Xiaoou, ZHANG Hepeng, et al. Measuring crowd collectiveness[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(8): 1586–1599. doi: 10.1109/TPAMI.2014.2300484
[4]	DIREKOGLU C, SAH M, and O’CONNOR N E. Abnormal crowd behavior detection using novel optical flow-based features[C]. The 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italia, 2017: 1–6.
[5]	蒋俊, 张卓君, 高明亮, 等. 一种基于脉线流卷积神经网络的人群异常行为检测算法[J]. 工程科学与技术, 2020, 52(6): 215–222. JIANG Jun, ZHANG Zhuojun, GAO Mingliang, et al. An abnormal crowd behavior detection method based on streak flow CNN[J]. Advanced Engineering Sciences, 2020, 52(6): 215–222.
[6]	王洪雁, 周梦星. 基于光流及轨迹的人群异常行为检测[J]. 吉林大学学报: 工学版, 2020, 50(6): 2229–2237. WANG Hongyan and ZHOU Mengxing. Crowd abnormal behavior detection based on optical flow and track[J]. Journal of Jilin University:Engineering and Technology Edition, 2020, 50(6): 2229–2237.
[7]	XIE Shaoci, ZHANG Xiaohong, and CAI Jing. Video crowd detection and abnormal behavior model detection based on machine learning method[J]. Neural Computing and Applications, 2019, 31(1): 175–184.
[8]	MEHRAN R, OYAMA A, and SHAH M. Abnormal crowd behavior detection using social force model[C]. 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, 2009: 935–942.
[9]	周培培, 丁庆海, 罗海波, 等. 视频监控中的人群异常行为检测与定位[J]. 光学学报, 2018, 38(8): 97–105. ZHOU Peipei, DING Qinghai, LUO Haibo, et al. Anomaly detection and location in crowded surveillance videos[J]. Acta Optica Sinica, 2018, 38(8): 97–105.
[10]	CAI Yiheng, LIU Jiaqi, GUO Yajun, et al. Video anomaly detection with multi-scale feature and temporal information fusion[J]. Neurocomputing, 2021, 423: 264–273. doi: 10.1016/j.neucom.2020.10.044
[11]	CONG Yang, YUAN Junsong, and LIU Ji. Abnormal event detection in crowded scenes using sparse representation[J]. Pattern Recognition, 2013, 46(7): 1851–1864. doi: 10.1016/j.patcog.2012.11.021
[12]	WU Shandong, MOORE B E, and SHAH M. Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes[C]. Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, USA, 2010: 2054–2060.
[13]	SALIGRAMA V and CHEN Zhu. Video anomaly detection based on local statistical aggregates[C]. Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, USA, 2012: 2112–2119.
[14]	ISOLA P, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks[C]. Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 1125–1134.
[15]	XIONG Guogang, CHENG Jun, WU Xinyu, et al. An energy model approach to people counting for abnormal crowd behavior detection[J]. Neurocomputing, 2012, 83: 121–135. doi: 10.1016/j.neucom.2011.12.007
[16]	彭月平, 蒋镕圻, 徐蕾. 基于C3D-GRNN模型的人群异常行为识别算法[J]. 测控技术, 2020, 39(7): 44–50. PENG Yueping, JIANG Rongqi, and XU Lei. An algorithm for identifying crowd abnormal behavior based on C3D-GRNN model[J]. Measurement &Control Technology, 2020, 39(7): 44–50.