Spiking Neural Network Recognition Method Based on Dynamic Visual Motion Features

DONG Junfei; JIANG Runhao; YAN Rui; TANG Huajin

doi:10.11999/JEIT221478

Volume 45 Issue 8

Aug. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 > 45(8): 2731-2738

DONG Junfei, JIANG Runhao, YAN Rui, TANG Huajin. Spiking Neural Network Recognition Method Based on Dynamic Visual Motion Features[J]. Journal of Electronics & Information Technology, 2023, 45(8): 2731-2738. doi: 10.11999/JEIT221478

Citation:

DONG Junfei, JIANG Runhao, YAN Rui, TANG Huajin. Spiking Neural Network Recognition Method Based on Dynamic Visual Motion Features[J]. Journal of Electronics & Information Technology, 2023, 45(8): 2731-2738. doi: 10.11999/JEIT221478

Citation:

PDF( 2216 KB)

Spiking Neural Network Recognition Method Based on Dynamic Visual Motion Features

doi: 10.11999/JEIT221478

1.
College of Computer Science, Sichuan University, Chengdu 610065, China
2.
College of Computer Science and Technology, Zhejiang University, Hangzhou 310058, China
3.
College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310014, China

Funds: The National Key Research and Development Program of China (2020AAA0105900)

Received Date: 2022-11-25
Rev Recd Date: 2023-05-02

Available Online: 2023-05-19

Publish Date: 2023-08-21

Abstract

Abstract

Considering the shortcomings of the low recognition accuracy and poor real-time performance of existing Spiking Neural Networks (SNN) for dynamic visual event streams, a SNN recognition method based on dynamic visual motion features is proposed in this paper. First, the dynamic motion features in the event stream are extracted using the event-based motion history information representation and gradient direction calculation. Then, the spatiotemporal pooling operation is introduced to eliminate the redundancy of events in the temporal and spatial domain, further retaining the significant motion features. Finally, the feature event streams are fed into the SNN for learning and recognition. Experiments conducted on benchmark dynamic visual datasets show that dynamic visual motion features can significantly improve the recognition accuracy and computational speed of SNN for event streams.
- Dynamic visual perception,
- Event camera,
- Spiking Neural Network (SNN),
- Action recognition,
- Motion feature extraction

FullText(HTML)

References(24)

References

[1]	GALLEGO G, DELBRÜCK T, ORCHARD G, et al. Event-based vision: A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(1): 154–180. doi: 10.1109/TPAMI.2020.3008413
[2]	张宇豪, 袁孟雯, 陆宇婧, 等. 面向动态事件流的神经网络转换方法[J]. 计算机应用, 2022, 42(10): 3033–3039. doi: 10.11772/j.issn.1001-9081.2021091607 ZHANG Yuhao, YUAN Mengwen, LU Yujing, et al. Neural network conversion method for dynamic event stream[J]. Journal of Computer Applications, 2022, 42(10): 3033–3039. doi: 10.11772/j.issn.1001-9081.2021091607
[3]	李家宁, 田永鸿. 神经形态视觉传感器的研究进展及应用综述[J]. 计算机学报, 2021, 44(6): 1258–1286. LI Jianing and TIAN Yonghong, Recent advances in neuromorphic vision sensors: A survey[J]. Chinese Journal of Computers, 2021, 44(6): 1258–1286.
[4]	ROY K, JAISWAL A, and PANDA P. Towards spike-based machine intelligence with neuromorphic computing[J]. Nature, 2019, 575(7784): 607–617. doi: 10.1038/s41586-019-1677-2
[5]	张铁林, 徐波. 脉冲神经网络研究现状及展望[J]. 计算机学报, 2021, 44(9): 1767–1785. ZHANG Tielin and XU Bo. Research advances and perspectives on spiking neural networks[J]. Chinese Journal of Computers, 2021, 44(9): 1767–1785.
[6]	胡一凡, 李国齐, 吴郁杰, 等. 脉冲神经网络研究进展综述[J]. 控制与决策, 2021, 36(1): 1–26. doi: 10.13195/j.kzyjc.2020.1006 HU Yifan, LI Guoqi, WU Yujie, et al. Spiking neural networks: A survey on recent advances and new directions[J]. Control and Decision, 2021, 36(1): 1–26. doi: 10.13195/j.kzyjc.2020.1006
[7]	ORCHARD G, MEYER C, ETIENNE-CUMMINGS R, et al. HFirst: A temporal approach to object recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(10): 2028–2040. doi: 10.1109/TPAMI.2015.2392947
[8]	ZHAO Bo, DING Ruoxi, CHEN Shoushun, et al. Feedforward categorization on AER motion events using cortex-like features in a spiking neural network[J]. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(9): 1963–1978. doi: 10.1109/TNNLS.2014.2362542
[9]	XIAO Rong, TANG Huajin, MA Yuhao, et al. An event-driven categorization model for AER image sensors using multispike encoding and learning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(9): 3649–3657. doi: 10.1109/TNNLS.2019.2945630
[10]	LIU Qianhui, PAN Gang, RUAN Haibo, et al. Unsupervised AER object recognition based on multiscale spatio-temporal features and spiking neurons[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(12): 5300–5311. doi: 10.1109/TNNLS.2020.2966058
[11]	LIU Qianhui, RUAN Haibo, XING Dong, et al. Effective AER object classification using segmented probability-maximization learning in spiking neural networks[C]. The 34th AAAI Conference on Artificial Intelligence, New York, USA, 2020: 1308–1315.
[12]	HE Weihua, WU Yujie, DENG Lei, et al. Comparing SNNs and RNNs on neuromorphic vision datasets: Similarities and differences[J]. Neural Networks, 2020, 132: 108–120. doi: 10.1016/j.neunet.2020.08.001
[13]	SHRESTHA S B and ORCHARD G. SLAYER: Spike layer error reassignment in time[C]. The 32nd International Conference on Neural Information Processing Systems, Montreal, Canada, 2018: 1419–1428.
[14]	KAISER J, MOSTAFA H, and NEFTCI E. Synaptic plasticity dynamics for deep continuous local learning (DECOLLE)[J]. Frontiers in Neuroscience, 2020, 14: 424. doi: 10.3389/fnins.2020.00424
[15]	LIU Qianhui, XING Dong, TANG Huajin, et al. Event-based action recognition using motion information and spiking neural networks[C]. The Thirtieth International Joint Conference on Artificial Intelligence, Montreal, Canada, 2021: 1743–1749.
[16]	AFSHAR S, HAMILTON T J, TAPSON J, et al. Investigation of event-based surfaces for high-speed detection, unsupervised feature extraction, and object recognition[J]. Frontiers in Neuroscience, 2019, 12: 1047. doi: 10.3389/fnins.2018.01047
[17]	MANDERSCHEID J, SIRONI A, BOURDIS N, et al. Speed invariant time surface for learning to detect corner points with event-based cameras[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 10237–10246.
[18]	ZHAO Bo, YU Qiang, DING Ruoxi, et al. Event-driven simulation of the tempotron spiking neuron[C]. 2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings, Lausanne, Switzerland, 2014: 667–670.
[19]	AMIR A, TABA B, BERG D, et al. A low power, fully event-based gesture recognition system[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 7388–7397.
[20]	MIAO Shu, CHEN Guang, NING Xiangyu, et al. Neuromorphic vision datasets for pedestrian detection, action recognition, and fall detection[J]. Frontiers in Neurorobotics, 2019, 13: 38. doi: 10.3389/fnbot.2019.00038
[21]	XING Yannan, DI CATERINA G, and SORAGHAN J. A new spiking convolutional recurrent neural network (SCRNN) with applications to event-based hand gesture recognition[J]. Frontiers in Neuroscience, 2020, 14: 590164. doi: 10.3389/fnins.2020.590164
[22]	GEORGE A M, BANERJEE D, DEY S, et al. A reservoir-based convolutional spiking neural network for gesture recognition from DVS input[C]. 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, United Kingdom, 2020: 1–9.
[23]	LU Junwei, DONG Junfei, YAN Rui, et al. An event-based categorization model using spatio-temporal features in a spiking neural network[C]. 2020 12th International Conference on Advanced Computational Intelligence (ICACI), Dali, China, 2020: 385–390.
[24]	LAGORCE X, ORCHARD G, GALLUPPI F, et al. HOTS: A hierarchy of event-based time-surfaces for pattern recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(7): 1346–1359. doi: 10.1109/TPAMI.2016.2574707