Spiking Neural Network Robot Tactile Object Recognition Method with Regularization Constraints

YANG Jing; JI Xiaoyang; LI Shaobo; HU Jianjun; WANG Yang; LIU Tingqing

doi:10.11999/JEIT220711

Volume 45 Issue 7

Jul. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 > 45(7): 2595-2604

YANG Jing, JI Xiaoyang, LI Shaobo, HU Jianjun, WANG Yang, LIU Tingqing. Spiking Neural Network Robot Tactile Object Recognition Method with Regularization Constraints[J]. Journal of Electronics & Information Technology, 2023, 45(7): 2595-2604. doi: 10.11999/JEIT220711

Citation:

YANG Jing, JI Xiaoyang, LI Shaobo, HU Jianjun, WANG Yang, LIU Tingqing. Spiking Neural Network Robot Tactile Object Recognition Method with Regularization Constraints[J]. Journal of Electronics & Information Technology, 2023, 45(7): 2595-2604. doi: 10.11999/JEIT220711

Citation:

PDF( 8001 KB)

Spiking Neural Network Robot Tactile Object Recognition Method with Regularization Constraints

doi: 10.11999/JEIT220711

YANG Jing^{1, 2, 3},
JI Xiaoyang^{1, 2
,
,},
LI Shaobo^{1, 2, 3},
HU Jianjun⁴,
WANG Yang^{1, 2},
LIU Tingqing^{1, 2}

1.
State Key Laboratory of Public Big Data, Guizhou University, Guiyang 550025, China
2.
College of Computer Science and Technology, Guizhou University, Guiyang 550025, China
3.
Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guiyang 550025, China
4.
Department of Computer Science and Engineering, University of South Carolina, Columbia 29208, USA

Funds: The National Key R&D Program of China(2018AAA0101800), The National Natural Science Foundation of China (62166005), The Joint Open Fund Project of Key Laboratories of the Ministry of Education ([2020]245), The Guizhou University Talents Project (GRJH[2020]09), The Natural Science Foudation of Guizhou Province (QKH-ZK[2022]130, QKH[2021]335)

Received Date: 2022-06-01
Rev Recd Date: 2022-07-30

Available Online: 2022-08-19

Publish Date: 2023-07-10

Abstract

Abstract

It is important for the future development of intelligent robots to expand tactile perception ability, which determines the scope of application scenarios for robots. Tactile data collected by tactile sensors are the basis of robotics work, but these data have complex spatio-temporal properties. Spiking neural network has rich spatio-temporal dynamics and event-driven nature. It can better process spatio-temporal information and be applied to artificial intelligence chips to bring higher energy efficiency to robots. To solve the problem of backpropagation failure in the network training process caused by the discreteness of neuron spike activity in the spiking neural network, from the perspective of the dynamic system of the intelligent robot, the spiking activity approximation function is introduced to make the spiking neural network back-propagation gradient descent method effective. The over-fitting problem caused by the small amount of tactile data is alleviated by the regularization methods. Finally, the spiking neural network robot tactile object recognition algorithm SnnTd and SnnTdlc with regularization constraints are proposed. Compared with the classical methods TactileSGNet, Grid-based CNN, MLP and GCN, the SnnTd method tactile object recognition rate is improved by 5.00% over the best method TactileSGNet on EvTouch-Containers dataset, and the SnnTdlc method tactile object recognition rate is improved by 3.16% over the best method TactileSGNet on EvTouch-Objects dataset.
- Tactile perception,
- Spiking neural network,
- Recognition algorithm,
- Regularization method,
- Back propagation

FullText(HTML)

References(33)

References

[1]	LI Qiang, KROEMER O, SU Zhe, et al. A review of tactile information: Perception and action through touch[J]. IEEE Transactions on Robotics, 2020, 36(6): 1619–1634. doi: 10.1109/TRO.2020.3003230
[2]	WANG Chunfeng, DONG Lin, PENG Dengfeng, et al. Tactile sensors for advanced intelligent systems[J]. Advanced Intelligent Systems, 2019, 1(8): 1900090. doi: 10.1002/aisy.201900090
[3]	DAHIYA R S, METTA G, VALLE M, et al. Tactile sensing—from humans to humanoids[J]. IEEE Transactions on Robotics, 2010, 26(1): 1–20. doi: 10.1109/TRO.2009.2033627
[4]	AHMADI R, PACKIRISAMY M, DARGAHI J, et al. Discretely loaded beam-type optical fiber tactile sensor for tissue manipulation and palpation in minimally invasive robotic surgery[J]. IEEE Sensors Journal, 2012, 12(1): 22–32. doi: 10.1109/JSEN.2011.2113394
[5]	YEH S K, HSIEH M L, and FANG W. CMOS-based tactile force sensor: A review[J]. IEEE Sensors Journal, 2021, 21(11): 12563–12577. doi: 10.1109/JSEN.2021.3060539
[6]	程龙, 刘泽宇. 柔性触觉传感技术及其在医疗康复机器人的应用[J]. 控制与决策, 2022, 37(6): 1409–1432. doi: 10.13195/j.kzyjc.2021.1896 CHENG Long and LIU Zeyu. Flexible tactile sensing technology and its application in medical rehabilitation robots[J]. Control and Decision, 2022, 37(6): 1409–1432. doi: 10.13195/j.kzyjc.2021.1896
[7]	DAHIYA R, YOGESWARAN N, LIU Fengyuan, et al. Large-area soft e-skin: The challenges beyond sensor designs[J]. Proceedings of the IEEE, 2019, 107(10): 2016–2033. doi: 10.1109/JPROC.2019.2941366
[8]	SUNDARAM S, KELLNHOFER P, LI Yunzhu, et al. Learning the signatures of the human grasp using a scalable tactile glove[J]. Nature, 2019, 569(7758): 698–702. doi: 10.1038/s41586-019-1234-z
[9]	KIM K, SIM M, LIM S H, et al. Tactile avatar: Tactile sensing system mimicking human tactile cognition[J]. Advanced Science, 2021, 8(7): 2002362. doi: 10.1002/advs.202002362
[10]	ZHU Minglu, SUN Zhongda, ZHANG Zixuan, et al. Haptic-feedback smart glove as a creative human-machine interface (HMI) for virtual/augmented reality applications[J]. Science Advances, 2020, 6(19): z8693. doi: 10.1126/sciadv.aaz8693
[11]	YAN Youcan, HU Zhe, YANG Zhengbao, et al. Soft magnetic skin for super-resolution tactile sensing with force self-decoupling[J]. Science Robotics, 2021, 6(51): c8801. doi: 10.1126/scirobotics.abc8801
[12]	BARTOLOZZI C, ROS P M, DIOTALEVI F, et al. Event-driven encoding of off-the-shelf tactile sensors for compression and latency optimisation for robotic skin[C]. 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Vancouver, Canada, 2017: 166–173.
[13]	DAVIES M, WILD A, ORCHARD G, et al. Advancing neuromorphic computing with Loihi: A survey of results and outlook[J]. Proceedings of the IEEE, 2021, 109(5): 911–934. doi: 10.1109/JPROC.2021.3067593
[14]	DAVIES M, SRINIVASA N, LIN T H, et al. Loihi: A neuromorphic manycore processor with on-chip learning[J]. IEEE Micro, 2018, 38(1): 82–99. doi: 10.1109/MM.2018.112130359
[15]	胡一凡, 李国齐, 吴郁杰, 等. 脉冲神经网络研究进展综述[J]. 控制与决策, 2021, 36(1): 1–26. doi: 10.13195/j.kzyjc.2020.1006 HU Yifan, LI Guoqi, WU Yujie, et al. Spiking neural networks A survey on recent advances and new directions[J]. Control and Decision, 2021, 36(1): 1–26. doi: 10.13195/j.kzyjc.2020.1006
[16]	张铁林, 徐波. 脉冲神经网络研究现状及展望[J]. 计算机学报, 2021, 44(9): 1767–1785. doi: 10.11897/SP.J.1016.2021.01767 ZHANG Tielin and XU Bo. Research advances and perspectives on spiking neural networks[J]. Chinese Journal of Computers, 2021, 44(9): 1767–1785. doi: 10.11897/SP.J.1016.2021.01767
[17]	BARBIER T, TEULIÈRE C, and TRIESCH J. Spike timing-based unsupervised learning of orientation, disparity, and motion representations in a spiking neural network[C]. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 1377–1386.
[18]	PAREDES-VALLÉS F, SCHEPER K Y, and DE CROON G C. Unsupervised learning of a hierarchical spiking neural network for optical flow estimation: From events to global motion perception[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2051–2064. doi: 10.1109/TPAMI.2019.2903179
[19]	HAN Bing, SRINIVASAN G, and ROY K. RMP-SNN: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[C]. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 13555–13564.
[20]	DENG Lei, WU Yujie, HU Xing, et al. Rethinking the performance comparison between SNNS and ANNS[J]. Neural Networks, 2020, 121: 294–307. doi: 10.1016/j.neunet.2019.09.005
[21]	LEE J H, DELBRUCK T, and PFEIFFER M. Training deep spiking neural networks using backpropagation[J]. Frontiers in Neuroscience, 2016, 10: 508. doi: 10.3389/fnins.2016.00508
[22]	WU Yujie, DENG Lei, LI Guoqi, et al. Direct training for spiking neural networks: Faster, larger, better[C]. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, Honolulu, USA, 2019: 1311–1318.
[23]	SEE H H, LIM B, LI Si, et al. ST-MNIST-the spiking tactile MNIST neuromorphic dataset[J]. arXiv: 2005.04319, 2020.
[24]	TAUNYAZOV T, CHUA Yansong, GAO Ruihan, et al. Fast texture classification using tactile neural coding and spiking neural network[C]. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, USA, IEEE, 2020: 9890–9895.
[25]	TAUNYAZOV T, SNG W, SEE H H, et al. Event-driven visual-tactile sensing and learning for robots[J]. arXiv: 2009.07083, 2020.
[26]	GU Fuqaing, SNG Weicong, TAUNYAZOV T, et al. TactileSGNet: A spiking graph neural network for event-based tactile object recognition[C]. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, USA, 2020: 9876–9882.
[27]	SUSHKO V, GALL J, and KHOREVA A. One-shot GAN: Learning to generate samples from single images and videos[C]. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 2596–2600.
[28]	程曦, 张志勇. 基于人工神经网络的复杂介质中波的传播不确定性分析方法[J]. 电子与信息学报, 2021, 43(12): 3662–3670. doi: 10.11999/JEIT200755 CHENG Xi and ZHANG Zhiyong. An uncertainty analysis method of wave propagation in complex media based on artificial neural network[J]. Journal of Electronics &Information Technology, 2021, 43(12): 3662–3670. doi: 10.11999/JEIT200755
[29]	曹毅, 费鸿博, 李平, 等. 基于多流卷积和数据增强的声场景分类方法[J]. 华中科技大学学报(自然科学版), 2022, 50(4): 40–46. doi: 10.13245/j.hust.220407 CAO Yi, FEI Hongbo, LI Ping, et al. Acoustic scene classification method based on multi-stream convolution and data augmentation[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2022, 50(4): 40–46. doi: 10.13245/j.hust.220407
[30]	GAL Y and GHAHRAMANI Z. Dropout as a Bayesian approximation: representing model uncertainty in deep learning[C]. Proceedings of the 33rd International Conference on Machine Learning, New York, USA, 2016: 1050–1059.
[31]	李翔, 陈硕, 杨健. 泛化界正则项: 理解权重衰减正则形式的统一视角[J]. 计算机学报, 2021, 44(10): 2122–2134. doi: 10.11897/SP.J.1016.2021.02122 LI Xiang, CHEN Shuo, and YANG Jian. Generalization bound Regularizer: A unified perspective for understanding weight decay[J]. Chinese Journal of Computers, 2021, 44(10): 2122–2134. doi: 10.11897/SP.J.1016.2021.02122
[32]	KINGMA D P and BA J. Adam: A method for stochastic optimization[J]. arXiv: 1412.6980v9, 2014.
[33]	LOSHCHILOV I and HUTTER F. SGDR: Stochastic gradient descent with warm restarts[J]. arXiv: 1608.03983v3, 2016.