基于FMCW雷达的多维参数手势识别算法

王勇; 吴金君; 田增山; 周牧; 王沙沙

doi:10.11999/JEIT180485

基于FMCW雷达的多维参数手势识别算法

doi: 10.11999/JEIT180485

重庆邮电大学通信与信息工程学院重庆 400065

基金项目: 国家自然科学基金(61771083, 61704015)，长江学者和创新团队发展计划基金(IRT1299)，重庆市科委重点实验室专项经费基金，重庆市基础与前沿研究计划基金(cstc2017jcyjAX0380, cstc2015jcyjBX0065)，重庆市高校优秀成果转化基金(KJZH17117)，重庆市教委科学技术研究项目(KJ1704083)

详细信息

作者简介:
王勇：男，1987年生，讲师，研究方向为无线通信、能效优化、室内定位、深度学习理论等

吴金君：男，1994年生，硕士生，研究方向为手势识别和深度学习技术

田增山：男，1968年生，教授，博士生导师，研究方向为移动通信、个人通信、GPS及蜂窝网定位技术等

周牧：男，1984年生，教授，研究方向为无线定位与导航技术、信号侦察与检测技术、凸优化与深度学习理论等

王沙沙：女，1992年生，硕士生，研究方向为深度学习技术和雷达信号处理

通讯作者:
吴金君　xnwujj@foxmail.com

中图分类号: TN958; TN98
计量
- 文章访问数: 3355
- HTML全文浏览量: 1155
- PDF下载量: 227
- 被引次数: 0
出版历程
- 收稿日期: 2018-05-21
- 修回日期: 2018-08-30
- 网络出版日期: 2018-09-13
- 刊出日期: 2019-04-01

Gesture Recognition with Multi-dimensional Parameter Using FMCW Radar

School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Funds: The National Natural Science Foundation of China (61771083, 61704015), The Program for Changjiang Scholars and Innovative Research Team in University (IRT1299), The Special Fund of Chongqing Key Laboratory (CSTC), The Fundamental and Frontier Research Project of Chongqing (cstc2017jcyjAX0380, cstc2015jcyjBX0065), The University Outstanding Achievement Transformation Project of Chongqing (KJZH17117), The Scientific and Technological Research Foundation of Chongqing Municipal Education Commission (KJ1704083)

摘要

摘要:
该文提出一种基于调频连续波(FMCW)雷达多维参数的卷积神经网络手势识别方法。通过对雷达信号进行时频分析，估计手势目标的距离、多普勒和角度参数，构建出手势动作的多维参数数据集。同时，为了进行手势特征提取和精确分类，提出多分支网络结构和高维特征融合的方案，设计出具有端到端结构的RDA-T多维参数卷积神经网络。实验结果表明，结合手势动作的距离、多普勒和角度信息进行多维参数学习，所提方法有效解决了单维参数手势识别方法中手势描述信息量低的问题，且手势识别准确率相较于单参数方法提高了5%～8%。
- FMCW雷达 /
- 手势识别 /
- 深度学习 /
- 卷积神经网络
Abstract:
A multi-parameter convolutional neural network method is proposed for gesture recognition based on Frequency Modulated Continuous Wave (FMCW) radar. A multidimensional parameter dataset is constructed for gestures by performing time-frequency analysis of the radar signal to estimate the distance, Doppler and angle parameters of the gesture target. To realize feature extraction and classification accurately, an end-to-end structured Range-Doppler-Angle of Time (RDA-T) multi-dimensional parameter convolutional neural network scheme is further proposed using multi-branch network structure and high-dimensional feature fusion. The experimental results reveal that using the combined gestures information of distance, Doppler and angle for multi-parameter learning, the proposed scheme resolves the problem of low information quantity of single-dimensional gesture recognition methods, and its accuracy outperforms the single-dimensional methods in terms of gesture recognition by 5%～8%.
- FMCW radar /
- Gesture recognition /
- Deeplearning /
- Convolutional Neural Network (CNN)

HTML全文

图 1 数据集中6种手势对应RTM, DTM和ATM

下载: 全尺寸图片幻灯片

图 2 RDA-T卷积神经网络结构图

下载: 全尺寸图片幻灯片

图 3 RTM, DTM和ATM准确率曲线

下载: 全尺寸图片幻灯片

图 4 fc6, fc7层不同尺寸准确率比较

下载: 全尺寸图片幻灯片

图 5 RDA-T网络不同初始学习率准确率曲线

下载: 全尺寸图片幻灯片

图 6 不同学习衰减率交叉熵损失值比较

下载: 全尺寸图片幻灯片

图 7 多维参数数据集上不同方法准确率曲线比较

下载: 全尺寸图片幻灯片

表 1 不同帧数下的准确率比较结果(%)

	8帧数据集	16帧数据集	24帧数据集	32帧数据集
准确率	70.3	79.7	88.3	95.3

下载: 导出CSV

表 2 手势分类混淆矩阵(%)

		预测类别
		前推	后拉	左滑	右滑	前后推拉	左右滑动
真实类别	前推	90	0	4	0	0	6
	后拉	0	100	0	0	0	0
	左滑	0	0	98	2	0	0
	右滑	0	2	0	94	0	4
	前后推拉	4	0	0	0	94	2
	左右滑动	2	0	0	0	2	96

下载: 导出CSV

表 3 本文方法与其他方法准确率和算法复杂度对比

网络结构	数据集	平均准确率(%)	空间复杂度(10⁶ Byte)	时间复杂度(10⁹ FLOPS)	单个样本处理时间(ms)
CNN^[9]单分支网络	RTM	89.6	9.7	1.51	2.9
	DTM	87.3
	ATM	84.3
VGG16-Net^[14]单分支网络	RTM	89.3	136.0	15.5	3.9
	DTM	86.3
	ATM	87.0
本文单分支网络(单参数网络)	RTM	90.6	64.1	0.74	2.9
	DTM	88.3
	ATM	89.3
CNN^[9]多分支网络	多维参数数据集	91.6	26.6	4.50	5.6
VGG16-Net^[14]多分支网络		92.0	362.1	46.16	10.3
本文多分支网络(RDA-T网络)		95.3	89.6	2.11	3.9

下载: 导出CSV

参考文献(23)

LI Yunan, MIAO Qiguang, TIAN Kuan, et al. Large-scale gesture recognition with a fusion of RGB-D data based on the C3D model[C]. 2016 23rd International Conference on Pattern Recognition, Cancun, Mexico, 2016: 25–30.

HE Yiwen, YANG Jianyu, SHAO Zhanpeng, et al. Salient feature point selection for real time RGB-D hand gesture recognition[C]. IEEE International Conference on Real-time Computing and Robotics, Okinawa, Japan, 2017: 103–108.

ALMASRE M A and AL-NUAIM H. Recognizing Arabic Sign Language gestures using depth sensors and a KSVM classifie[C]. Computer Science and Electronic Engineering, Colchester, UK, 2016: 146–151.

AUGUSTAUSKAS R and LIPNICKAS A. Robust hand detection using arm segmentation from depth data and static palm gesture recognition[C]. Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, Bucharest, Romania, 2017, 2: 664–667.

DEKKER B, JACOBS S, KOSSEN A S, et al. Gesture recognition with a low power FMCW radar and a deep convolutional neural network[C]. Radar Conference, Nuremberg, Germany, 2017: 163–166.

MOLCHANOV P, GUPTA S, KIM K, et al. Multi-sensor system for driver's hand-gesture recognition[C]. Automatic Face and Gesture Recognition, Ljubljana, Slovenia, 2015, 1: 1–8.

LIN J J, LI Yuanping, HSU W C, et al. Design of an FMCW radar baseband signal processing system for automotive application[J]. Springerplus, 2016, 5(1): 42–57 doi: 10.1186/s40064-015-1583-5

LI Gang, ZHANG Rui, RITCHIE M, et al. Sparsity-driven micro-Doppler feature extraction for dynamic hand gesture recognition[J]. IEEE Transactions on Aerospace & Electronic Systems, 2018, 54(2): 655–665 doi: 10.1109/TAES.2017.2761229

ZHANG Shimeng, LI Gang, RITCHIE M, et al. Dynamic hand gesture classification based on radar micro-Doppler signatures[C]. 2016 CIE International Conference on Radar (RADAR), Guangzhou, China, 2016: 1–4.

KIM Y and TOOMAJIAN B. Hand gesture recognition using micro-Doppler signatures with convolutional neural network[J]. IEEE Access, 2016, 4: 7125–7130 doi: 10.1109/ACCESS.2016.2617282

MOLCHANOV P, GUPTA S, KIM K, et al. Short-range FMCW monopulse radar for hand-gesture sensing[C]. Radar Conference, Arlington, USA, 2015: 1491–1496.

WANG Saiwen, SONG Jie, LIEN J, et al. Interacting with soli: Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum[C]. Proceedings of the 29th Annual Symposium on User Interface Software and Technology, New York, USA, 2016: 851–860.

KRIZHEVSKY A, SUTSKEVER I, and HINTON G E. Imagenet classification with deep convolutional neural networks[C]. Advances in Neural Information Processing Systems, Montreal, Canada, 2012: 1097–1105.

SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[OL]. https://arxiv.org/abs/1409. 1556, 2014.

HE Kaiming, ZHANG Xianyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778.

SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]. Computer Vision and Pattern Recognition, Boston, USA, 2015: 1–9.

IOFFE S and SZEGEDY C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[OL]. https://arxiv.org/abs/1502.03167, 2015.

SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]. Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 2818–2826.

TRAN D, BOURDEV L, FERGUS R, et al. Learning spatiotemporal features with 3D convolutional networks[C]. International Conference on Computer Vision, Santiago, Chile, 2015: 4489–4497.

MOLCHANOV P, GUPTA S, KIM K, et al. Hand gesture recognition with 3D convolutional neural networks[C]. Computer Vision and Pattern Recognition Workshops, Boston, USA, 2015: 1–7.

SIMONYAN K and ZISSERMAN A. Two-stream convolutional networks for action recognition in videos[C]. Advances in Neural Information Processing Systems, Montreal, Canada, 2014: 568–576.

SCHMIDT R. Multiple emitter location and signal parameter estimation[J]. IEEE Transactions on Antennas and Propagation, 1986, 34(3): 276–280 doi: 10.1109/TAP.1986.1143830

HE Kaiming and SUN Jian. Convolutional neural networks at constrained time cost[C]. IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 5353–5360.

施引文献

资源附件(0)

访问统计