具有正则化约束的脉冲神经网络机器人触觉物体识别方法

杨静; 吉晓阳; 李少波; 胡建军; 王阳; 刘庭卿

doi:10.11999/JEIT220711

具有正则化约束的脉冲神经网络机器人触觉物体识别方法

doi: 10.11999/JEIT220711

杨静^{1, 2, 3},
吉晓阳^{1, 2, ,},
李少波^{1, 2, 3},
胡建军⁴,
王阳^{1, 2},
刘庭卿^{1, 2}

1.
贵州大学省部共建公共大数据国家重点实验室贵阳 550025
2.
贵州大学计算机科学与技术学院贵阳 550025
3.
贵州大学现代制造技术教育部重点实验室贵阳 550025
4.
美国南卡罗莱纳州立大学计算机科学与工程系哥伦比亚 29208

基金项目: 国家重点研发计划(2018AAA0101800)，国家自然科学基金(62166005)，教育部重点实验室开放项目(黔教合KY字[2020]245)，贵州省高层次留学人才项目((2021)09号)，贵州省自然科学基金项目(黔科合基础-ZK[2022]一般130，黔科合支撑[2021]335)

详细信息

作者简介:
杨静：男，副教授，研究方向为触觉感知

吉晓阳：男，硕士生，研究方向为触觉感知

李少波：男，教授，研究方向为制造大数据

胡建军：男，教授，研究方向为智能分析

王阳：男，博士后，研究方向为人工智能基础理论

刘庭卿：男，硕士生，研究方向为触觉感知

通讯作者:
吉晓阳　gs.xyji20@gzu.edu.cn

中图分类号: TP183
计量
- 文章访问数: 1022
- HTML全文浏览量: 552
- PDF下载量: 178
- 被引次数: 30
出版历程
- 收稿日期: 2022-06-01
- 修回日期: 2022-07-30
- 网络出版日期: 2022-08-19
- 刊出日期: 2023-07-10

Spiking Neural Network Robot Tactile Object Recognition Method with Regularization Constraints

YANG Jing^{1, 2, 3},
JI Xiaoyang^{1, 2
, ,},
LI Shaobo^{1, 2, 3},
HU Jianjun⁴,
WANG Yang^{1, 2},
LIU Tingqing^{1, 2}

1.
State Key Laboratory of Public Big Data, Guizhou University, Guiyang 550025, China
2.
College of Computer Science and Technology, Guizhou University, Guiyang 550025, China
3.
Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guiyang 550025, China
4.
Department of Computer Science and Engineering, University of South Carolina, Columbia 29208, USA

Funds: The National Key R&D Program of China(2018AAA0101800), The National Natural Science Foundation of China (62166005), The Joint Open Fund Project of Key Laboratories of the Ministry of Education ([2020]245), The Guizhou University Talents Project (GRJH[2020]09), The Natural Science Foudation of Guizhou Province (QKH-ZK[2022]130, QKH[2021]335)

摘要

摘要: 拓展触觉感知能力是智能机器人未来发展的重要方向之一，决定着机器人的应用场景范围。由触觉传感器采集的数据是机器人完成触觉感知任务基础，但触觉数据具有复杂的时空性。脉冲神经网络具有丰富的时空动力学特征和契合硬件的事件驱动性，能更好地处理时空信息和应用于人工智能芯片给机器人带来更高能效。该文针对脉冲神经网络神经元脉冲活动离散性导致网络训练过程反向传播失效的问题，从智能触觉机器人动态系统角度，引入脉冲活动近似函数使脉冲神经网络反向传播梯度下降法有效；针对触觉脉冲数据量少导致的过拟合问题，融合正则化方法加以缓解；最后，提出具有正则化约束的脉冲神经网络机器人触觉物体识别(Spiking neural network Tactile dropout, SnnTd; Spiking neural network Tactile dropout-l2-cosine annealing, SnnTdlc)算法。相较于经典方法TactileSGNet, Grid-based CNN, MLP和GCN, SnnTd正则化方法触觉物体识别率在EvTouch-Containers数据集上比最好方法TactileSGNet提升了5.00%，SnnTdlc正则化方法触觉物体识别率在EvTouch-Objects数据集上比最好方法TactileSGNet提升了3.16%。
- 触觉感知 /
- 脉冲神经网络 /
- 识别算法 /
- 正则化方法 /
- 反向传播
Abstract: It is important for the future development of intelligent robots to expand tactile perception ability, which determines the scope of application scenarios for robots. Tactile data collected by tactile sensors are the basis of robotics work, but these data have complex spatio-temporal properties. Spiking neural network has rich spatio-temporal dynamics and event-driven nature. It can better process spatio-temporal information and be applied to artificial intelligence chips to bring higher energy efficiency to robots. To solve the problem of backpropagation failure in the network training process caused by the discreteness of neuron spike activity in the spiking neural network, from the perspective of the dynamic system of the intelligent robot, the spiking activity approximation function is introduced to make the spiking neural network back-propagation gradient descent method effective. The over-fitting problem caused by the small amount of tactile data is alleviated by the regularization methods. Finally, the spiking neural network robot tactile object recognition algorithm SnnTd and SnnTdlc with regularization constraints are proposed. Compared with the classical methods TactileSGNet, Grid-based CNN, MLP and GCN, the SnnTd method tactile object recognition rate is improved by 5.00% over the best method TactileSGNet on EvTouch-Containers dataset, and the SnnTdlc method tactile object recognition rate is improved by 3.16% over the best method TactileSGNet on EvTouch-Objects dataset.
- Tactile perception /
- Spiking neural network /
- Recognition algorithm /
- Regularization method /
- Back propagation

HTML全文

图 1 网络模型结构图

下载: 全尺寸图片幻灯片

图 2 LIF神经元脉冲活动

下载: 全尺寸图片幻灯片

图 3 脉冲神经网络的空域和时域传播过程

下载: 全尺寸图片幻灯片

图 4 LIF神经元中的Dropout

下载: 全尺寸图片幻灯片

图 5 EvTouch-Containers数据集Loss值变化情况

下载: 全尺寸图片幻灯片

图 6 Acc, TrainingLoss 和 TestLoss 10 次实验统计结果

下载: 全尺寸图片幻灯片

图 7 EvTouch-Objects 数据集 Loss 值变化情况

下载: 全尺寸图片幻灯片

图 8 Acc, TrainingLoss 和 TestLoss 10 次实验结果

下载: 全尺寸图片幻灯片

图 9 正则化模型在不同数据集上的混淆矩阵

下载: 全尺寸图片幻灯片

算法1　具有L2正则化约束的脉冲神经网络机器人触觉物体识别算法
相关参数：学习率 ${\mu _t}$ , 网络参数 $w$ 和 $b$ ，时间步长 $T$ ，触觉脉冲数　　　　　　据 $S = \left\{ {{S_1},{S_2},\cdots,{S_T}} \right\}$ ，目标输出　　　　　　 $R = \left\{ {{R_1},{R_2},\cdots,{R_T}} \right\}$ ，权重衰减参数 $\sigma$ ，损失函数　　　　　　 $L = L(R,S) + \dfrac{\sigma }{2}{\left\\| w \right\\|^2}$
初始化： $w$ ， $b$ ，空列表 $K = \{ \}$
for ${\rm{Epoch}} = 1,2, \cdots ,N$ :
for $t = 1,2, \cdots ,T$ :
(1) 输入 ${S_t}$ 到脉冲神经网络，获得输出脉冲 ${K_t}$ ，将 ${K_t}$ 添　　　　　加到 $K$ ；
end
(2) 计算损失 $L = L(R,S) + \dfrac{\sigma }{2}{\left\\| w \right\\|^2}$ ；
(3) 计算 $\dfrac{{\partial \left(L + \dfrac{\sigma }{2}{{\left\\| w \right\\|}^2}\right)}}{{\partial w}} = \displaystyle\sum\nolimits_{t = 1}^T {\dfrac{{\partial L}}{{\partial P_t^n}}E_t^{n - 1}} + \sigma w$ , 　　　　　 $\dfrac{{\partial (L + \dfrac{\sigma }{2}{{\left\\| w \right\\|}^2})}}{{\partial b}} = \displaystyle\sum\nolimits_{t = 1}^T {\dfrac{{\partial L}}{{\partial P_t^n}}}$ ；
(4) 根据Adam优化算法进行更新;
(5) 通过余弦退火算法更新学习率 ${\mu _t}$ ；
(6) 更新参数 $w$ ， $b$ ；
end

下载: 导出CSV

表 1 超参数设置表

参数	值	参数	值
Epochs	100	宽度系数 $a$	0.5
初始学习率 ${\rm{lr}}$	1×10^–3	dropout1	0.2
学习率衰减因子 $\alpha$	0.1	dropout2	0.5
学习率衰减代数 ${\rm{lrEpochs}}$	40	动量 ${\beta _1}$	0.9
余弦退火最小学习率 ${{\rm{lr}}_{ {\text{min} } } }$	5×10^–6	均方根传播 ${\beta _2}$	0.999
膜电位阈值 ${P_{{\text{TH}}}}$	0.5	权重衰减 $\sigma$	0.01

下载: 导出CSV

表 2 EvTouch-Containers数据集实验结果

模型	方法			Acc(%)	TrainingLoss	TestLoss
模型	Dropout	L2	Cosine Annealing	Acc(%)	TrainingLoss	TestLoss
SnnT				66.00±0.86	2.13±0.04	1.03±0.01
SnnTd	√			69.17±1.18	2.61±0.01	1.04±0.01
SnnTl		√		68.33±1.11	2.13±0.02	1.02±0.01
SnnTc			√	67.17±1.58	1.96±0.01	1.03±0.01
SnnTdl	√	√		66.33±1.32	2.67±0.01	1.04±0.01
SnnTdc	√		√	66.17±0.81	2.45±0.01	1.03±0.01
SnnTlc		√	√	67.17±1.37	2.01±0.01	1.02±0.01
SnnTdlc	√	√	√	67.33±0.86	2.51±0.01	1.04±0.01

下载: 导出CSV

表 3 EvTouch-Objects数据集实验结果

模型	方法			Acc(%)	TrainingLoss	TestLoss
模型	Dropout	L2	Cosine Annealing	Acc(%)	TrainingLoss	TestLoss
SnnT				89.17±0.36	1.60±0.01	1.13±0.01
SnnTd	√			90.63±0.59	2.34±0.01	1.20±0.01
SnnTl		√		89.31±0.88	1.71±0.02	1.14±0.01
SnnTc			√	89.10±0.57	1.48±0.01	1.12±0.01
SnnTdl	√	√		90.90±0.39	2.38±0.01	1.20±0.01
SnnTdc	√		√	90.49±0.33	2.17±0.01	1.18±0.01
SnnTlc		√	√	89.86±0.67	1.58±0.02	1.12±0.01
SnnTdlc	√	√	√	91.04±0.39	2.22±0.01	1.18±0.01

下载: 导出CSV

表 4 SnnTdlc模型与经典方法在两个数据集下Acc的实验结果(%)

模型	EvTouch-Containers	EvTouch-Objects
TactileSGNet	64.17±2.75	89.44±0.55
Grid-based CNN	60.17±2.78	88.40±1.14
MLP	58.83±2.49	85.97±0.85
GCN	58.83±2.84	85.14±1.51
SnnTd	69.17±1.18	90.63±0.59
SnnTdlc	67.33±0.86	91.04±0.39

下载: 导出CSV

参考文献(33)

[1]	LI Qiang, KROEMER O, SU Zhe, et al. A review of tactile information: Perception and action through touch[J]. IEEE Transactions on Robotics, 2020, 36(6): 1619–1634. doi: 10.1109/TRO.2020.3003230
[2]	WANG Chunfeng, DONG Lin, PENG Dengfeng, et al. Tactile sensors for advanced intelligent systems[J]. Advanced Intelligent Systems, 2019, 1(8): 1900090. doi: 10.1002/aisy.201900090
[3]	DAHIYA R S, METTA G, VALLE M, et al. Tactile sensing—from humans to humanoids[J]. IEEE Transactions on Robotics, 2010, 26(1): 1–20. doi: 10.1109/TRO.2009.2033627
[4]	AHMADI R, PACKIRISAMY M, DARGAHI J, et al. Discretely loaded beam-type optical fiber tactile sensor for tissue manipulation and palpation in minimally invasive robotic surgery[J]. IEEE Sensors Journal, 2012, 12(1): 22–32. doi: 10.1109/JSEN.2011.2113394
[5]	YEH S K, HSIEH M L, and FANG W. CMOS-based tactile force sensor: A review[J]. IEEE Sensors Journal, 2021, 21(11): 12563–12577. doi: 10.1109/JSEN.2021.3060539
[6]	程龙, 刘泽宇. 柔性触觉传感技术及其在医疗康复机器人的应用[J]. 控制与决策, 2022, 37(6): 1409–1432. doi: 10.13195/j.kzyjc.2021.1896 CHENG Long and LIU Zeyu. Flexible tactile sensing technology and its application in medical rehabilitation robots[J]. Control and Decision, 2022, 37(6): 1409–1432. doi: 10.13195/j.kzyjc.2021.1896
[7]	DAHIYA R, YOGESWARAN N, LIU Fengyuan, et al. Large-area soft e-skin: The challenges beyond sensor designs[J]. Proceedings of the IEEE, 2019, 107(10): 2016–2033. doi: 10.1109/JPROC.2019.2941366
[8]	SUNDARAM S, KELLNHOFER P, LI Yunzhu, et al. Learning the signatures of the human grasp using a scalable tactile glove[J]. Nature, 2019, 569(7758): 698–702. doi: 10.1038/s41586-019-1234-z
[9]	KIM K, SIM M, LIM S H, et al. Tactile avatar: Tactile sensing system mimicking human tactile cognition[J]. Advanced Science, 2021, 8(7): 2002362. doi: 10.1002/advs.202002362
[10]	ZHU Minglu, SUN Zhongda, ZHANG Zixuan, et al. Haptic-feedback smart glove as a creative human-machine interface (HMI) for virtual/augmented reality applications[J]. Science Advances, 2020, 6(19): z8693. doi: 10.1126/sciadv.aaz8693
[11]	YAN Youcan, HU Zhe, YANG Zhengbao, et al. Soft magnetic skin for super-resolution tactile sensing with force self-decoupling[J]. Science Robotics, 2021, 6(51): c8801. doi: 10.1126/scirobotics.abc8801
[12]	BARTOLOZZI C, ROS P M, DIOTALEVI F, et al. Event-driven encoding of off-the-shelf tactile sensors for compression and latency optimisation for robotic skin[C]. 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Vancouver, Canada, 2017: 166–173.
[13]	DAVIES M, WILD A, ORCHARD G, et al. Advancing neuromorphic computing with Loihi: A survey of results and outlook[J]. Proceedings of the IEEE, 2021, 109(5): 911–934. doi: 10.1109/JPROC.2021.3067593
[14]	DAVIES M, SRINIVASA N, LIN T H, et al. Loihi: A neuromorphic manycore processor with on-chip learning[J]. IEEE Micro, 2018, 38(1): 82–99. doi: 10.1109/MM.2018.112130359
[15]	胡一凡, 李国齐, 吴郁杰, 等. 脉冲神经网络研究进展综述[J]. 控制与决策, 2021, 36(1): 1–26. doi: 10.13195/j.kzyjc.2020.1006 HU Yifan, LI Guoqi, WU Yujie, et al. Spiking neural networks A survey on recent advances and new directions[J]. Control and Decision, 2021, 36(1): 1–26. doi: 10.13195/j.kzyjc.2020.1006
[16]	张铁林, 徐波. 脉冲神经网络研究现状及展望[J]. 计算机学报, 2021, 44(9): 1767–1785. doi: 10.11897/SP.J.1016.2021.01767 ZHANG Tielin and XU Bo. Research advances and perspectives on spiking neural networks[J]. Chinese Journal of Computers, 2021, 44(9): 1767–1785. doi: 10.11897/SP.J.1016.2021.01767
[17]	BARBIER T, TEULIÈRE C, and TRIESCH J. Spike timing-based unsupervised learning of orientation, disparity, and motion representations in a spiking neural network[C]. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 1377–1386.
[18]	PAREDES-VALLÉS F, SCHEPER K Y, and DE CROON G C. Unsupervised learning of a hierarchical spiking neural network for optical flow estimation: From events to global motion perception[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2051–2064. doi: 10.1109/TPAMI.2019.2903179
[19]	HAN Bing, SRINIVASAN G, and ROY K. RMP-SNN: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[C]. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 13555–13564.
[20]	DENG Lei, WU Yujie, HU Xing, et al. Rethinking the performance comparison between SNNS and ANNS[J]. Neural Networks, 2020, 121: 294–307. doi: 10.1016/j.neunet.2019.09.005
[21]	LEE J H, DELBRUCK T, and PFEIFFER M. Training deep spiking neural networks using backpropagation[J]. Frontiers in Neuroscience, 2016, 10: 508. doi: 10.3389/fnins.2016.00508
[22]	WU Yujie, DENG Lei, LI Guoqi, et al. Direct training for spiking neural networks: Faster, larger, better[C]. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, Honolulu, USA, 2019: 1311–1318.
[23]	SEE H H, LIM B, LI Si, et al. ST-MNIST-the spiking tactile MNIST neuromorphic dataset[J]. arXiv: 2005.04319, 2020.
[24]	TAUNYAZOV T, CHUA Yansong, GAO Ruihan, et al. Fast texture classification using tactile neural coding and spiking neural network[C]. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, USA, IEEE, 2020: 9890–9895.
[25]	TAUNYAZOV T, SNG W, SEE H H, et al. Event-driven visual-tactile sensing and learning for robots[J]. arXiv: 2009.07083, 2020.
[26]	GU Fuqaing, SNG Weicong, TAUNYAZOV T, et al. TactileSGNet: A spiking graph neural network for event-based tactile object recognition[C]. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, USA, 2020: 9876–9882.
[27]	SUSHKO V, GALL J, and KHOREVA A. One-shot GAN: Learning to generate samples from single images and videos[C]. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 2596–2600.
[28]	程曦, 张志勇. 基于人工神经网络的复杂介质中波的传播不确定性分析方法[J]. 电子与信息学报, 2021, 43(12): 3662–3670. doi: 10.11999/JEIT200755 CHENG Xi and ZHANG Zhiyong. An uncertainty analysis method of wave propagation in complex media based on artificial neural network[J]. Journal of Electronics &Information Technology, 2021, 43(12): 3662–3670. doi: 10.11999/JEIT200755
[29]	曹毅, 费鸿博, 李平, 等. 基于多流卷积和数据增强的声场景分类方法[J]. 华中科技大学学报(自然科学版), 2022, 50(4): 40–46. doi: 10.13245/j.hust.220407 CAO Yi, FEI Hongbo, LI Ping, et al. Acoustic scene classification method based on multi-stream convolution and data augmentation[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2022, 50(4): 40–46. doi: 10.13245/j.hust.220407
[30]	GAL Y and GHAHRAMANI Z. Dropout as a Bayesian approximation: representing model uncertainty in deep learning[C]. Proceedings of the 33rd International Conference on Machine Learning, New York, USA, 2016: 1050–1059.
[31]	李翔, 陈硕, 杨健. 泛化界正则项: 理解权重衰减正则形式的统一视角[J]. 计算机学报, 2021, 44(10): 2122–2134. doi: 10.11897/SP.J.1016.2021.02122 LI Xiang, CHEN Shuo, and YANG Jian. Generalization bound Regularizer: A unified perspective for understanding weight decay[J]. Chinese Journal of Computers, 2021, 44(10): 2122–2134. doi: 10.11897/SP.J.1016.2021.02122
[32]	KINGMA D P and BA J. Adam: A method for stochastic optimization[J]. arXiv: 1412.6980v9, 2014.
[33]	LOSHCHILOV I and HUTTER F. SGDR: Stochastic gradient descent with warm restarts[J]. arXiv: 1608.03983v3, 2016.

施引文献

期刊类型引用(12)

1.	王广龙，田杰，朱文杰，方丹. 基于特征匹配的非刚性大位移光流算法. 北京理工大学学报. 2020(04): 421-426+440 . 百度学术
2.	刘晨，张龙波，王雷，卢海涛. 基于超像素重建的多尺度B样条医学图像配准. 智能计算机与应用. 2019(01): 24-27 . 百度学术
3.	何凯，闫佳星，魏颖，王阳. 基于改进光流场模型的非刚性图像配准. 天津大学学报(自然科学与工程技术版). 2018(05): 491-496 . 百度学术
4.	张静亚，李周雁. 有限元弹性配准中的驱动外力及其网格细化. 常熟理工学院学报. 2018(05): 73-79 . 百度学术
5.	华亮，程天宇，顾菊平，俞钶安. 基于ROI及Clifford代数相对不变量的3D医学图像配准. 图学学报. 2017(01): 90-96 . 百度学术
6.	杨飒，夏明华，郑志硕. 基于多项式确定性矩阵的SIFT医学图像配准算法. 激光与光电子学进展. 2016(08): 128-134 . 百度学术
7.	于荷峰，吕晓琪，黄显武，贾东征. 基于改进Demons算法的三维肺部医学影像配准研究. 计算机应用研究. 2016(04): 1269-1272 . 百度学术
8.	张静亚，王加俊. 一种改进的非刚性医学图像配准算法. 计算机应用研究. 2015(04): 1261-1264 . 百度学术
9.	杨飒，郑志硕. 基于稀疏随机投影的SIFT医学图像配准算法. 量子电子学报. 2015(03): 283-289 . 百度学术
10.	华亮，黄宇，丁立军，冯浩，顾菊平. Clifford代数空间上的三维多模医学图像配准. 光电工程. 2014(01): 65-72 . 百度学术
11.	李京娜，邓嘉兴，王刚. 卫星图像配准及匹配曲线特征评估法. 光电工程. 2014(03): 73-81 . 百度学术
12.	华亮，丁立军，黄宇，冯浩，顾菊平. Clifford代数几何不变量3D医学图像配准的方法. 计算机科学. 2014(06): 304-308 . 百度学术