Millimeter Wave Radar Gesture Recognition Algorithm Based on Spatio-temporal Compression Feature Representation Learning

HAN Chong; HAN Lei; SUN Lijuan; GUO Jian

doi:10.11999/JEIT211221

Volume 44 Issue 4

Apr. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(4): 1274-1283

HAN Chong, HAN Lei, SUN Lijuan, GUO Jian. Millimeter Wave Radar Gesture Recognition Algorithm Based on Spatio-temporal Compression Feature Representation Learning[J]. Journal of Electronics & Information Technology, 2022, 44(4): 1274-1283. doi: 10.11999/JEIT211221

Citation:

HAN Chong, HAN Lei, SUN Lijuan, GUO Jian. Millimeter Wave Radar Gesture Recognition Algorithm Based on Spatio-temporal Compression Feature Representation Learning[J]. Journal of Electronics & Information Technology, 2022, 44(4): 1274-1283. doi: 10.11999/JEIT211221

Citation:

PDF( 3197 KB)

Millimeter Wave Radar Gesture Recognition Algorithm Based on Spatio-temporal Compression Feature Representation Learning

doi: 10.11999/JEIT211221

HAN Chong^{1, 2
,
,},
HAN Lei¹,
SUN Lijuan^{1, 2},
GUO Jian^{1, 2}

1.
College of Computer, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
2.
Jiangsu High Technology Research Key Laboratory for Wireless Sensor Networks, Nanjing 210003, China

Funds: The National Natural Science Foundation of China (61873131, 61872194, 61902237)

Received Date: 2021-11-04
Accepted Date: 2022-02-21
Rev Recd Date: 2022-02-20

Available Online: 2022-03-05

Publish Date: 2022-04-18

Abstract

Abstract

To solve the problems of data preprocessing and feature utilization in the existing work of gesture recognition of radio frequency signals, a gesture recognition algorithm for spatio-temporal compressed feature representation learning of Frequency Modulated Continuous Wave (FMCW) millimeter wave radar is proposed. First, static interference removal and moving target point filtering are performed on the Range-Doppler (RD) image of the FMCW radar echo reflected by the hand, which could reduce the interference of clutter on the gesture signal, and also reduce greatly the calculation of the data. Then, a method for compressing the spatial-temporal features of gesture is adopted to realize the compression mapping of multidimensional features using the dominant velocity of the moving target point to represent the motion characteristics of the gesture, which includes the key feature information of the gesture motion. Finally, a single channel Convolutional Neural Network (CNN) is designed to learn and classify multidimensional gesture feature information in multi-user and multi-location gesture application scenes. Experimental results show that the proposed gesture recognition method has significant performance in recognition accuracy, real-time performance and generalization ability.
- Millimeter wave radar,
- Gesture recognition,
- Feature compression,
- Convolutional Neural Network(CNN)

FullText(HTML)

References(20)

References

[1]	CHENG Hong, DAI Zhongjun, and LIU Zicheng. Image-to-class dynamic time warping for 3D hand gesture recognition[C]. 2013 IEEE International Conference on Multimedia and Expo, San Jose, USA, 2013: 1–6.
[2]	LI Yi. Hand gesture recognition using Kinect[C]. 2012 IEEE International Conference on Computer Science and Automation Engineering, Beijing, China, 2012: 196–199.
[3]	REN Zhou, YUAN Junsong, MENG Jingjing, et al. Robust part-based hand gesture recognition using kinect sensor[J]. IEEE Transactions on Multimedia, 2013, 15(5): 1110–1120. doi: 10.1109/TMM.2013.2246148
[4]	SAHA S, GHOSH S, KONAR A, et al. Gesture recognition from Indian classical dance using kinect sensor[C]. The 5th International Conference on Computational Intelligence, Communication Systems and Networks, Madrid, Spain, 2013: 3–8.
[5]	LING Yu, Chen Xiang, RUAN Yuwen, et al. Comparative study of gesture recognition based on accelerometer and photoplethysmography sensor for gesture interactions in wearable devices[J]. IEEE Sensors Journal, 2021, 21(15): 17107–17117. doi: 10.1109/JSEN.2021.3081714
[6]	JIANG Xianta, MERHI L K, XIAO Zhengang, et al. Exploration of force myography and surface electromyography in hand gesture classification[J]. Medical Engineering & Physics, 2017, 41: 63–73. doi: 10.1016/J.MEDENGPHY.2017.01.015
[7]	LIU Jingtao, GU Changzhan, ZHANG Yueping, et al. Analysis on a 77 GHz MIMO radar for touchless gesture sensing[J]. IEEE Sensors Letters, 2020, 4(5): 3500804. doi: 10.1109/LSENS.2020.2987814
[8]	LI Gang, ZHANG Rui, RITCHIE M, et al. Sparsity-based dynamic hand gesture recognition using micro-Doppler signatures[C]. 2017 IEEE Radar Conference, Seattle, USA, 2017: 928–931.
[9]	LIEN J, GILLIAN N, KARAGOZLER M E, et al. Soli: Ubiquitous gesture sensing with millimeter wave radar[J]. ACM Transactions on Graphics, 2016, 35(4): 142. doi: 10.1145/2897824.2925953
[10]	ZHANG Jiajun, TAO Jinkun, and SHI Zhiguo. Doppler-radar based hand gesture recognition system using convolutional neural networks[C]. 2017 International Conference in Communications, Signal Processing, and Systems, Harbin, China, 2017: 1096–1113.
[11]	MOLCHANOV P, GUPTA S, KIM K, et al. Multi-sensor system for driver's hand-gesture recognition[C]. The 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition. Ljubljana, Slovenia, 2015: 1–8.
[12]	ZHENG Qiangwen, YANG Lijie, XIE Yaping, et al. A target detection scheme with decreased complexity and enhanced performance for range-Doppler FMCW radar[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 70: 8001113. doi: 10.1109/TIM.2020.3027407
[13]	SUN Yuliang, FEI Tai, LI Xibo, et al. Multi-feature encoder for radar-based gesture recognition[C]. 2020 IEEE International Radar Conference, Washington, USA, 2020: 351–356.
[14]	SUN Yuliang, FEI Tai, LI Xibo, et al. Real-time radar-based gesture detection and recognition built in an edge-computing platform[J]. IEEE Sensors Journal, 2020, 20(18): 10706–10716. doi: 10.1109/JSEN.2020.2994292
[15]	XIA Zhaoyang, LUOMEI Yixiang, ZHOU Chenglong, et al. Multidimensional feature representation and learning for robust hand-gesture recognition on commercial millimeter-wave radar[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(6): 4749–4764. doi: 10.1109/TGRS.2020.3010880
[16]	夏朝阳, 周成龙, 介钧誉, 等. 基于多通道调频连续波毫米波雷达的微动手势识别[J]. 电子与信息学报, 2020, 42(1): 164–172. doi: 10.11999/JEIT190797 XIA Zhaoyang, ZHOU Chenglong, JIE Junyu, et al. Micro-motion gesture recognition based on multi-channel frequency modulated continuous wave millimeter wave radar[J]. Journal of Electronics &Information Technology, 2020, 42(1): 164–172. doi: 10.11999/JEIT190797
[17]	KARPATHY A, TODERICI G, SHETTY S, et al. Large-scale video classification with convolutional neural networks[C]. 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, USA, 2014: 1725–1732.
[18]	TRAN D, BOURDEV L, FERGUS R, et al. Learning spatiotemporal features with 3D convolutional networks[C]. 2015 IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 4489–4497.
[19]	WANG Xuanhan, GAO Lianli, SONG Jingkuan, et al. Beyond frame-level CNN: Saliency-aware 3-D CNN with LSTM for video action recognition[J]. IEEE Signal Processing Letters, 2017, 24(4): 510–514. doi: 10.1109/LSP.2016.2611485
[20]	WANG Saiwen, SONG Jie, LIEN J, et al. Interacting with soli: Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum[C]. The 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan, 2016: 851–860.