High-precision Gesture Recognition Based on DenseNet and Convolutional Block Attention Module

ZHAO Yaqin; SONG Yuqing; WU Han; HE Shengyang; LIU Puqiu; WU Longwen

doi:10.11999/JEIT230165

Volume 46 Issue 3

Mar. 2024

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2024 > 46(3): 967-976

ZHAO Yaqin, SONG Yuqing, WU Han, HE Shengyang, LIU Puqiu, WU Longwen. High-precision Gesture Recognition Based on DenseNet and Convolutional Block Attention Module[J]. Journal of Electronics & Information Technology, 2024, 46(3): 967-976. doi: 10.11999/JEIT230165

Citation:

ZHAO Yaqin, SONG Yuqing, WU Han, HE Shengyang, LIU Puqiu, WU Longwen. High-precision Gesture Recognition Based on DenseNet and Convolutional Block Attention Module[J]. Journal of Electronics & Information Technology, 2024, 46(3): 967-976. doi: 10.11999/JEIT230165

Citation:

PDF( 4831 KB)

High-precision Gesture Recognition Based on DenseNet and Convolutional Block Attention Module

doi: 10.11999/JEIT230165

1.
School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China
2.
China Aerospace Science & Industry Corporation 8511 company, Nanjing 211100, China

Funds: The National Natural Science Foundation of China (61671185, 62071153)

Received Date: 2023-03-16
Rev Recd Date: 2023-09-07

Available Online: 2023-09-12

Publish Date: 2024-03-27

Abstract

Abstract

Non-contact gesture recognition is a new type of human-computer interaction method with broad application prospects. It can be used in Augmented Reality (AR)/Virtual Reality (VR), smart homes, smart medical, etc., and has recently become a research hotspot. Motivated by the need for more precise micro-motion gesture recognition using mm-wave radar in recent years, a novel micro-motion gesture recognition method based on MIMO millimeter wave radar is proposed in this paper. The MilliMeter Wave CAScaded (MMWCAS) radar cascaded with four AWR1243 radar boards is used to collect gesture echoes. Time-frequency analysis is performed on gesture echoes, and the hand target is detected based on the Range-Doppler (RD) map and 3D point cloud. Through data pre-processing, the Range-Time Map (RTM), Doppler-Time Map (DTM), Azimuth-Time Map (ATM) and Elevation-Time Map (ETM) of the gestures are extracted to more comprehensively characterize the motion of the hand gesture. The mixed Feature-Time Maps (FTM) are formed and adopted for the recognition of 12 types of micro-motion gestures. An innovative gesture recognition network based on DenseNet and Convolutional Block Attention Module (CBAM) is designed, and the mixed FTM is used as the input of the network. Experimental results show that the recognition accuracy reaches 99.03%, achieving high-accuracy gesture recognition. It is discovered that the network focuses on the first half of the gesture movement, which improves the recognition accuracy.
- Gesture recognition,
- Millimeter wave radar,
- Convolutional Neural Network(CNN),
- Convolutional Block Attention Module(CBAM)

FullText(HTML)

References(30)

References

[1]	VERDADERO M S, MARTINEZ-OJEDA C O, and CRUZ J C D. Hand gesture recognition system as an alternative interface for remote controlled home appliances[C]. The 10th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management, Baguio, Philippines, 2018: 1–5.
[2]	WISENER W J, RODRIGUEZ J D, OVANDO A, et al. A top-view hand gesture recognition system for IoT applications[C]. The 5th International Conference on Smart Systems and Inventive Technology, Tirunelveli, India, 2023: 430–434.
[3]	KIM K M and CHOI J I. Passengers’ gesture recognition model in self-driving vehicles: Gesture recognition model of the passengers’ obstruction of the vision of the driver[C]. The 4th International Conference on Computer and Communication Systems, Singapore, 2019: 239–242.
[4]	NOORUDDIN N, DEMBANI R, and MAITLO N. HGR: Hand-gesture-recognition based text input method for AR/VR wearable devices[C]. 2020 IEEE International Conference on Systems, Man, and Cybernetics, Toronto, Canada, 2020: 744–751.
[5]	LIU Zhenyu, LIU Haoming, and MA Chongrun. A robust hand gesture sensing and recognition based on dual-flow fusion with FMCW radar[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 4028105. doi: 10.1109/LGRS.2022.3217390.
[6]	ZHANG Wenjin, WANG Jiacun, and LAN Fangping. Dynamic hand gesture recognition based on short-term sampling neural networks[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 8(1): 110–120. doi: 10.1109/JAS.2020.1003465.
[7]	LEÓN D G, GRÖLI J, YEDURI S R, et al. Video hand gestures recognition using depth camera and lightweight CNN[J]. IEEE Sensors Journal, 2022, 22(14): 14610–14619. doi: 10.1109/JSEN.2022.3181518.
[8]	LIEN J, GILLIAN N, KARAGOZLER M E, et al. Soli: Ubiquitous gesture sensing with millimeter wave radar[J]. ACM Transactions on Graphics, 2016, 35(4): 142. doi: 10.1145/2897824.2925953.
[9]	WANG Saiwen, SONG Jie, LIEN J, et al. Interacting with soli: Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum[C]. The 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan, 2016: 851–860.
[10]	SHEN Xiangyu, ZHENG Haifeng, FENG Xinxin, et al. ML-HGR-Net: A meta-learning network for FMCW radar based hand gesture recognition[J]. IEEE Sensors Journal, 2022, 22(11): 10808–10817. doi: 10.1109/JSEN.2022.3169231.
[11]	ZHANG Xuhao, WU Qisong, and ZHAO Dixian. Dynamic hand gesture recognition using FMCW radar sensor for driving assistance[C]. 2018 10th International Conference on Wireless Communications and Signal Processing, Hangzhou, China, 2018: 1–6.
[12]	YU J T, YEN L, and TSENG P H. mmWave radar-based hand gesture recognition using range-angle image[C]. 2020 IEEE 91st Vehicular Technology Conference, Antwerp, Belgium, 2020: 1–5.
[13]	LIU Haipeng, ZHOU Anfu, DONG Zihe, et al. M-Gesture: Person-independent real-time in-air gesture recognition using commodity millimeter wave radar[J]. IEEE Internet of Things Journal, 2022, 9(5): 3397–3415. doi: 10.1109/JIOT.2021.3098338.
[14]	SMITH J W, THIAGARAJAN S, WILLIS R, et al. Improved static hand gesture classification on deep convolutional neural networks using novel sterile training technique[J]. IEEE Access, 2021, 9: 10893–10902. doi: 10.1109/ACCESS.2021.3051454.
[15]	GAN Liangyu, LIU Yuan, LI Yanzhong, et al. Gesture recognition system using 24 GHz FMCW radar sensor realized on real-time edge computing platform[J]. IEEE Sensors Journal, 2022, 22(9): 8904–8914. doi: 10.1109/JSEN.2022.3163449.
[16]	王勇, 吴金君, 田增山, 等. 基于FMCW雷达的多维参数手势识别算法[J]. 电子与信息学报, 2019, 41(4): 822–829. doi: 10.11999/JEIT180485. WANG Yong, WU Jinjun, TIAN Zengshan, et al. Gesture recognition with multi-dimensional parameter using FMCW radar[J]. Journal of Electronics &Information Technology, 2019, 41(4): 822–829. doi: 10.11999/JEIT180485.
[17]	王勇, 王沙沙, 田增山, 等. 基于FMCW雷达的双流融合神经网络手势识别方法[J]. 电子学报, 2019, 47(7): 1408–1415. doi: 10.3969/j.issn.0372-2112.2019.07.003. WANG Yong, WANG Shasha, TIAN Zengshan, et al. Two-stream fusion neural network approach for hand gesture recognition based on FMCW radar[J]. Acta Electronica Sinica, 2019, 47(7): 1408–1415. doi: 10.3969/j.issn.0372-2112.2019.07.003.
[18]	AHMED S, KIM W, PARK J, et al. Radar-based air-writing gesture recognition using a novel multistream CNN approach[J]. IEEE Internet of Things Journal, 2022, 9(23): 23869–23880. doi: 10.1109/JIOT.2022.3189395.
[19]	ALIREZAZAD K and MAURER L. FMCW radar-based hand gesture recognition using dual-stream CNN-GRU model[C]. 2022 24th International Microwave and Radar Conference, Gdansk, Poland, 2022: 1–5.
[20]	PARK G, CHANDRASEGAR V K, PARK J, et al. Increasing accuracy of hand gesture recognition using convolutional neural network[C]. 2022 International Conference on Artificial Intelligence in Information and Communication, Jeju Island, Korea, Republic of, 2022: 251–255.
[21]	DANG T L, NGUYEN H T, DAO D M, et al. SHAPE: A dataset for hand gesture recognition[J]. Neural Computing and Applications, 2022, 34(24): 21849–21862. doi: 10.1007/s00521-022-07651-1.
[22]	WANG Lingling, CHEN Xiaoyan, XIONG Wei, et al. Research on gesture recognition and classification based on attention mechanism[C]. 2022 IEEE 5th Advanced Information Management, Communicates, Electronic and Automation Control Conference, Chongqing, China, 2022: 1617–1621.
[23]	FANG Juan, XU Chao, WANG Chao, et al. Dynamic gesture recognition based on multimodal fusion model[C]. 2021 20th International Conference on Ubiquitous Computing and Communications, London, United Kingdom, 2021: 172–177.
[24]	GUO He, ZHANG Rui, LI Yang, et al. Research on human-vehicle gesture interaction technology based on computer visionbility[C]. 2022 IEEE 6th Advanced Information Technology, Electronic and Automation Control Conference, Beijing, China, 2022: 1161–1165.
[25]	STOVE A G. Linear FMCW radar techniques[J]. IEE Proceedings F (Radar and Signal Processing), 1992, 139(5): 343–350. doi: 10.1049/ip-f-2.1992.0048.
[26]	WINKLER V. Range Doppler detection for automotive FMCW radars[C]. 2007 European Microwave Conference, Munich, Germany, 2007: 166–169.
[27]	夏朝阳, 周成龙, 介钧誉, 等. 基于多通道调频连续波毫米波雷达的微动手势识别[J]. 电子与信息学报, 2020, 42(1): 164–172. doi: 10.11999/JEIT190797. XIA Zhaoyang, ZHOU Chenglong, JIE Junyu, et al. Micro-motion gesture recognition based on multi-channel frequency modulated continuous wave millimeter wave radar[J]. Journal of Electronics &Information Technology, 2020, 42(1): 164–172. doi: 10.11999/JEIT190797.
[28]	樊瑞宣, 姜高霞, 王文剑. 一种个性化k近邻的离群点检测算法[J]. 小型微型计算机系统, 2020, 41(4): 752–757. doi: 10.3969/j.issn.1000-1220.2020.04.014. FAN Ruixuan, JIANG Gaoxia, and WANG Wenjian. Outlier detection algorithm with personalized k-nearest neighbor[J]. Journal of Chinese Computer Systems, 2020, 41(4): 752–757. doi: 10.3969/j.issn.1000-1220.2020.04.014.
[29]	WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module[C]. The European Conference on Computer Vision, Munich, Germany, 2018: 3–19.
[30]	HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 2261–2269.