Handwriting Number Recognition Based on Millimeter-wave Radar with Dual-view Feature Fusion Network

FENG Xiang; LIU Tao; CUI Wenqing; WU Mufu; LI Fengcong; ZHAO Yinan

doi:10.11999/JEIT220687

Volume 45 Issue 6

Jun. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 > 45(6): 2134-2143

FENG Xiang, LIU Tao, CUI Wenqing, WU Mufu, LI Fengcong, ZHAO Yinan. Handwriting Number Recognition Based on Millimeter-wave Radar with Dual-view Feature Fusion Network[J]. Journal of Electronics & Information Technology, 2023, 45(6): 2134-2143. doi: 10.11999/JEIT220687

Citation:

FENG Xiang, LIU Tao, CUI Wenqing, WU Mufu, LI Fengcong, ZHAO Yinan. Handwriting Number Recognition Based on Millimeter-wave Radar with Dual-view Feature Fusion Network[J]. Journal of Electronics & Information Technology, 2023, 45(6): 2134-2143. doi: 10.11999/JEIT220687

Citation:

PDF( 7362 KB)

Handwriting Number Recognition Based on Millimeter-wave Radar with Dual-view Feature Fusion Network

doi: 10.11999/JEIT220687

School of Information Science and Engineering, Harbin Institute of Technology, Weihai 264209, China

Funds: The Natural Science Foundation of Shandong Province (ZR2019BF037)

Received Date: 2022-05-27
Rev Recd Date: 2022-10-05

Available Online: 2022-10-11

Publish Date: 2023-06-10

Abstract

Abstract

Against the epidemic background, the contactless human-computer interaction has great application prospects in the medical and health field. Among them, using gesture recognition method to realize non-contact instrument control is becoming the hotspot. To improve the robustness and accuracy, a method is proposed to realize the digital gesture recognition based on dual-view sequential feature fusion of millimeter-wave radars in this paper. Firstly, time series echo data of gesture numbers 0～9 from positive and side perspectives are collected synchronously. Secondly, datasets from different perspectives are preprocessed by implementing clutter suppression and data compression. Furthermore, the Attention embedded Dual View Fusion Network (ADVFNet) is constructed based on the intrinsic correlation of temporal features. Finally, using the collected dataset, the task of training network, fusing sequential feature, and recognizing digital gesture could be completed. Experimental results show that the recognition accuracy of proposed method is about 95%, which has faster network convergence and better model generalization ability compared with several existing methods. Moreover, the method could provide a new idea for future human-computer interaction of millimeter-wave radars.
- Millimeter-wave radar,
- Gesture number recognition,
- Dual-View Fusion Network (DVFNet),
- Attention mechanism

FullText(HTML)

References(23)

References

[1]	LI Xinyu, HE Yuan, FIORANELLI F, et al. Semisupervised human activity recognition with radar micro-Doppler signatures[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5103112. doi: 10.1109/TGRS.2021.3090106
[2]	EROL B and AMIN M G. Radar data cube processing for human activity recognition using multisubspace learning[J]. IEEE Transactions on Aerospace and Electronic Systems, 2019, 55(6): 3617–3628. doi: 10.1109/TAES.2019.2910980
[3]	冯心欣, 李文龙, 何兆, 等. 基于调频连续波雷达的多维信息特征融合人体姿势识别方法[J]. 电子与信息学报, 2022, 40(10): 3583–3591. doi: 10.11999JEIT220687 FENG Xinxin, LI Wenlong, HE Zhao, et al. Multi-dimensional information feature fusion for posture recognition based on frequency modulated continuous wave radar[J]. Journal of Electronics &Information Technology, 2022, 40(10): 3583–3591. doi: 10.11999JEIT220687
[4]	ZHANG Zhenyuan, TIAN Zengshan, and ZHOU Mu. Latern: Dynamic continuous hand gesture recognition using FMCW radar sensor[J]. IEEE Sensors Journal, 2018, 18(8): 3278–3289. doi: 10.1109/JSEN.2018.2808688
[5]	SKARIA S, AL-HOURANI A, LECH M, et al. Hand-gesture recognition using two-antenna Doppler radar with deep convolutional neural networks[J]. IEEE Sensors Journal, 2019, 19(8): 3041–3048. doi: 10.1109/JSEN.2019.2892073
[6]	HAZRA S and SANTRA A. Robust gesture recognition using millimetric-wave radar system[J]. IEEE Sensors Letters, 2018, 2(4): 1–4. doi: 10.1109/lsens.2018.2882642
[7]	FAN Teng, MA Chao, GU Zhitao, et al. Wireless hand gesture recognition based on continuous-wave Doppler radar sensors[J]. IEEE Transactions on Microwave Theory and Techniques, 2016, 64(11): 4012–4020. doi: 10.1109/TMTT.2016.2610427
[8]	CRISÓSTOMO DE CASTRO FILHO H, ABÍLIO DE CARVALHO JÚNIOR O, FERREIRA DE CARVALHO O L, et al. Rice crop detection using LSTM, Bi-LSTM, and machine learning models from Sentinel-1 time series[J]. Remote Sensing, 2020, 12(16): 2655. doi: 10.3390/rs12162655
[9]	SHRESTHA A, LI Haobo, LE KERNEC J, et al. Continuous human activity classification from FMCW radar with Bi-LSTM networks[J]. IEEE Sensors Journal, 2020, 20(22): 13607–13619. doi: 10.1109/JSEN.2020.3006386
[10]	SUN Yuliang, FEI Tai, LI Xibo, et al. Real-time radar-based gesture detection and recognition built in an edge-computing platform[J]. IEEE Sensors Journal, 2020, 20(18): 10706–10716. doi: 10.1109/JSEN.2020.2994292
[11]	王俊, 郑彤, 雷鹏, 等. 基于卷积神经网络的手势动作雷达识别方法[J]. 北京航空航天大学学报, 2018, 44(6): 1117–1123. doi: 10.13700/j.bh.1001-5965.2017.0397 WANG Jun, ZHENG Tong, LEI Peng, et al. Hand gesture recognition method by radar based on convolutional neural network[J]. Journal of Beijing University of Aeronautics and Astronautics, 2018, 44(6): 1117–1123. doi: 10.13700/j.bh.1001-5965.2017.0397
[12]	王勇, 王沙沙, 田增山, 等. 基于FMCW雷达的双流融合神经网络手势识别方法[J]. 电子学报, 2019, 47(7): 1408–1415. doi: 10.3969/j.issn.0372-2112.2019.07.003 WANG Yong, WANG Shasha, TIAN Zengshan, et al. Two-stream fusion neural network approach for hand gesture recognition based on FMCW radar[J]. Acta Electronica Sinica, 2019, 47(7): 1408–1415. doi: 10.3969/j.issn.0372-2112.2019.07.003
[13]	王勇, 吴金君, 田增山, 等. 基于FMCW雷达的多维参数手势识别算法[J]. 电子与信息学报, 2019, 41(4): 822–829. doi: 10.11999/JEIT180485 WANG Yong, WU Jinjun, TIAN Zengshan, et al. Gesture recognition with multi-dimensional parameter using FMCW radar[J]. Journal of Electronics &Information Technology, 2019, 41(4): 822–829. doi: 10.11999/JEIT180485
[14]	夏朝阳, 周成龙, 介钧誉, 等. 基于多通道调频连续波毫米波雷达的微动手势识别[J]. 电子与信息学报, 2020, 42(1): 164–172. doi: 10.11999/JEIT190797 XIA Zhaoyang, ZHOU Chenglong, JIE Junyu, et al. Micro-motion gesture recognition based on multi-channel frequency modulated continuous wave millimeter wave radar[J]. Journal of Electronics &Information Technology, 2020, 42(1): 164–172. doi: 10.11999/JEIT190797
[15]	靳标, 彭宇, 邝晓飞, 等. 基于串联式一维神经网络的毫米波雷达动态手势识别方法[J]. 电子与信息学报, 2021, 43(9): 2743–2750. doi: 10.11999/JEIT200894 JIN Biao, PENG Yu, KUANG Xiaofei, et al. Dynamic gesture recognition method based on millimeter-wave radar by one-dimensional series neural network[J]. Journal of Electronics &Information Technology, 2021, 43(9): 2743–2750. doi: 10.11999/JEIT200894
[16]	SHEN Xiangyu, ZHENG Haifeng, FENG Xinxin, et al. ML-HGR-Net: A meta-learning network for FMCW radar based hand gesture recognition[J]. IEEE Sensors Journal, 2022, 22(11): 10808–10817. doi: 10.1109/JSEN.2022.3169231
[17]	DONG Xichao, ZHAO Zewei, WANG Yupei, et al. FMCW radar-based hand gesture recognition using spatiotemporal deformable and context-aware convolutional 5-D feature representation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5107011. doi: 10.1109/TGRS.2021.3122332
[18]	RYU S J, SUH J S, BAEK S H, et al. Feature-based hand gesture recognition using an FMCW radar and its temporal feature analysis[J]. IEEE Sensors Journal, 2018, 18(18): 7593–7602. doi: 10.1109/JSEN.2018.2859815
[19]	邵正途, 许登荣, 徐文利, 等. 基于LSTM和残差网络的雷达有源干扰识别[J]. 系统工程与电子技术, 2023, 45(2): 416–423. SHAO Zhengtu, XU Dengrong, XU Wenli, et al. Radar active jamming recognition based on LSTM and residual network[J]. Systems Engineering and Electronics, 2023, 45(2): 416–423.
[20]	SHERSTINSKY A. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network[J]. Physica D:Nonlinear Phenomena, 2020, 404: 132306. doi: 10.1016/j.physd.2019.132306
[21]	JI Yuzhu, ZHANG Haijun, and WU Q M J. Salient object detection via multi-scale attention CNN[J]. Neurocomputing, 2018, 322: 130–140. doi: 10.1016/j.neucom.2018.09.061
[22]	LI Liu, XU Mai, LIU Hanruo, et al. A large-scale database and a CNN model for attention-based glaucoma detection[J]. IEEE Transactions on Medical Imaging, 2020, 39(2): 413–424. doi: 10.1109/TMI.2019.2927226
[23]	韩崇, 韩磊, 孙力娟, 等. 基于时空压缩特征表示学习的毫米波雷达手势识别算法[J]. 电子与信息学报, 2022, 44(4): 1274–1283. doi: 10.11999/JEIT211221 HAN Chong, HAN Lei. SUN Lijuan, et al. Millimeter wave radar gesture recognition algorithm based on Spatio-temporal compression feature representation learning[J]. Journal of Electronics &Information Technology, 2022, 44(4): 1274–1283. doi: 10.11999/JEIT211221