高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于DenseNet和卷积注意力模块的高精度手势识别

赵雅琴 宋雨晴 吴晗 何胜阳 刘璞秋 吴龙文

赵雅琴, 宋雨晴, 吴晗, 何胜阳, 刘璞秋, 吴龙文. 基于DenseNet和卷积注意力模块的高精度手势识别[J]. 电子与信息学报, 2024, 46(3): 967-976. doi: 10.11999/JEIT230165
引用本文: 赵雅琴, 宋雨晴, 吴晗, 何胜阳, 刘璞秋, 吴龙文. 基于DenseNet和卷积注意力模块的高精度手势识别[J]. 电子与信息学报, 2024, 46(3): 967-976. doi: 10.11999/JEIT230165
ZHAO Yaqin, SONG Yuqing, WU Han, HE Shengyang, LIU Puqiu, WU Longwen. High-precision Gesture Recognition Based on DenseNet and Convolutional Block Attention Module[J]. Journal of Electronics & Information Technology, 2024, 46(3): 967-976. doi: 10.11999/JEIT230165
Citation: ZHAO Yaqin, SONG Yuqing, WU Han, HE Shengyang, LIU Puqiu, WU Longwen. High-precision Gesture Recognition Based on DenseNet and Convolutional Block Attention Module[J]. Journal of Electronics & Information Technology, 2024, 46(3): 967-976. doi: 10.11999/JEIT230165

基于DenseNet和卷积注意力模块的高精度手势识别

doi: 10.11999/JEIT230165
基金项目: 国家自然科学基金(61671185, 62071153)
详细信息
    作者简介:

    赵雅琴:女,博士,教授,研究方向为辐射源识别、辐射源个体识别、无源定位、光通信、医学信号处理等

    宋雨晴:女,硕士生,研究方向为时频分析、信号处理和毫米波雷达等

    吴晗:男,硕士生,研究方向为时频分析、信号处理和毫米波雷达等

    何胜阳:男,博士,高级工程师,研究方向为无线光通信、嵌入式系统和算法加速等

    刘璞秋:男,硕士,研究方向为人工智能、时频分析、信号处理和毫米波雷达等

    吴龙文:男,博士,工程师,研究方向为辐射源识别、辐射源个体识别、无源定位、多核学习和医学信号处理等

    通讯作者:

    吴龙文 wulongwen@hit.edu.cn

  • 中图分类号: TN957.51

High-precision Gesture Recognition Based on DenseNet and Convolutional Block Attention Module

Funds: The National Natural Science Foundation of China (61671185, 62071153)
  • 摘要: 非接触的手势识别是一种新型人机交互方式,在增强现实(AR)/虚拟现实(VR)、智能家居、智能医疗等方面有着广阔的应用前景,近年来成为一个研究热点。由于需要利用毫米波雷达进行更精确的微动手势识别,该文提出一种新型的基于MIMO毫米波雷达的微动手势识别方法。采用4片AWR1243雷达板级联而成的毫米波级联(MMWCAS)雷达采集手势回波,对手势回波进行时频分析,基于距离-多普勒(RD)图和3D点云检测出人手目标。通过数据预处理,提取手势目标的距离-时间谱图(RTM)、多普勒-时间谱图(DTM)、方位角-时间谱图(ATM)和俯仰角-时间谱图(ETM),更加全面地表征手势的运动特征,并形成混合特征谱图(FTM),对12种微动手势进行识别。设计了基于DenseNet和卷积注意力模块的手势识别网络,将混合特征谱图作为网络的输入,创新性地融合了卷积注意力模块(CBAM),实验表明,识别准确率达到99.03%,且该网络将注意力放在手势动作的前半段,实现了高精度的手势识别。
  • 图  1  人手和雷达的相对位置示意图

    图  2  本文提出的手势识别方法示意图

    图  3  特征图谱提取过程

    图  4  所提手势识别网络的整体结构

    图  5  12种手势示意图

    图  6  12种手势的DTM示例

    图  7  12种手势的ATM示例

    图  8  12种手势的ETM示例

    图  9  所提网络的混淆矩阵

    图  10  注意力分布热图

    表  1  毫米波雷达参数设置

    参数名称 参数名称
    发射天线数12 调频带宽5 GHz
    接收天线数16调频斜率125 MHz/µs
    采集帧数28ADC采样率4000 ksps
    每帧chirp数128帧周期72 ms
    每chirp采样数128
    下载: 导出CSV

    表  2  实验平台

    属性型号/参数
    CPUR5-4600H
    GPUGTX1660TI
    内存16 GB
    操作系统Windows10 64 bit
    下载: 导出CSV

    表  3  各种CNN模型进行数据扩充的效果对比

    VGG16ResNet50ResNet101DenseNet121DenseNet161
    原始(%)96.5779.2195.0997.3697.68
    数据扩充之后(%)97.0891.7697.4598.1598.29
    计算复杂度单个batch计算量5,509,133,8241,558,820,8643,082,369,024956,748,8002,550,592,320
    网络参数个数134,308,55623,526,41242,518,5407,972,64828,671,688
    数据扩充后的单次迭代用时(s)73.4448.2574.6760.02149.58
    下载: 导出CSV

    表  4  CBAM在DenseNet121不同位置的效果对比(%)

    位置00000001001000110100010101100111
    识别率98.1597.8799.0398.4798.8098.4798.8997.96
    位置10001001101010111100110111101111
    识别率98.9398.1998.8097.4597.6997.7798.2897.91
    下载: 导出CSV
  • [1] VERDADERO M S, MARTINEZ-OJEDA C O, and CRUZ J C D. Hand gesture recognition system as an alternative interface for remote controlled home appliances[C]. The 10th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management, Baguio, Philippines, 2018: 1–5.
    [2] WISENER W J, RODRIGUEZ J D, OVANDO A, et al. A top-view hand gesture recognition system for IoT applications[C]. The 5th International Conference on Smart Systems and Inventive Technology, Tirunelveli, India, 2023: 430–434.
    [3] KIM K M and CHOI J I. Passengers’ gesture recognition model in self-driving vehicles: Gesture recognition model of the passengers’ obstruction of the vision of the driver[C]. The 4th International Conference on Computer and Communication Systems, Singapore, 2019: 239–242.
    [4] NOORUDDIN N, DEMBANI R, and MAITLO N. HGR: Hand-gesture-recognition based text input method for AR/VR wearable devices[C]. 2020 IEEE International Conference on Systems, Man, and Cybernetics, Toronto, Canada, 2020: 744–751.
    [5] LIU Zhenyu, LIU Haoming, and MA Chongrun. A robust hand gesture sensing and recognition based on dual-flow fusion with FMCW radar[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 4028105. doi: 10.1109/LGRS.2022.3217390.
    [6] ZHANG Wenjin, WANG Jiacun, and LAN Fangping. Dynamic hand gesture recognition based on short-term sampling neural networks[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 8(1): 110–120. doi: 10.1109/JAS.2020.1003465.
    [7] LEÓN D G, GRÖLI J, YEDURI S R, et al. Video hand gestures recognition using depth camera and lightweight CNN[J]. IEEE Sensors Journal, 2022, 22(14): 14610–14619. doi: 10.1109/JSEN.2022.3181518.
    [8] LIEN J, GILLIAN N, KARAGOZLER M E, et al. Soli: Ubiquitous gesture sensing with millimeter wave radar[J]. ACM Transactions on Graphics, 2016, 35(4): 142. doi: 10.1145/2897824.2925953.
    [9] WANG Saiwen, SONG Jie, LIEN J, et al. Interacting with soli: Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum[C]. The 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan, 2016: 851–860.
    [10] SHEN Xiangyu, ZHENG Haifeng, FENG Xinxin, et al. ML-HGR-Net: A meta-learning network for FMCW radar based hand gesture recognition[J]. IEEE Sensors Journal, 2022, 22(11): 10808–10817. doi: 10.1109/JSEN.2022.3169231.
    [11] ZHANG Xuhao, WU Qisong, and ZHAO Dixian. Dynamic hand gesture recognition using FMCW radar sensor for driving assistance[C]. 2018 10th International Conference on Wireless Communications and Signal Processing, Hangzhou, China, 2018: 1–6.
    [12] YU J T, YEN L, and TSENG P H. mmWave radar-based hand gesture recognition using range-angle image[C]. 2020 IEEE 91st Vehicular Technology Conference, Antwerp, Belgium, 2020: 1–5.
    [13] LIU Haipeng, ZHOU Anfu, DONG Zihe, et al. M-Gesture: Person-independent real-time in-air gesture recognition using commodity millimeter wave radar[J]. IEEE Internet of Things Journal, 2022, 9(5): 3397–3415. doi: 10.1109/JIOT.2021.3098338.
    [14] SMITH J W, THIAGARAJAN S, WILLIS R, et al. Improved static hand gesture classification on deep convolutional neural networks using novel sterile training technique[J]. IEEE Access, 2021, 9: 10893–10902. doi: 10.1109/ACCESS.2021.3051454.
    [15] GAN Liangyu, LIU Yuan, LI Yanzhong, et al. Gesture recognition system using 24 GHz FMCW radar sensor realized on real-time edge computing platform[J]. IEEE Sensors Journal, 2022, 22(9): 8904–8914. doi: 10.1109/JSEN.2022.3163449.
    [16] 王勇, 吴金君, 田增山, 等. 基于FMCW雷达的多维参数手势识别算法[J]. 电子与信息学报, 2019, 41(4): 822–829. doi: 10.11999/JEIT180485.

    WANG Yong, WU Jinjun, TIAN Zengshan, et al. Gesture recognition with multi-dimensional parameter using FMCW radar[J]. Journal of Electronics &Information Technology, 2019, 41(4): 822–829. doi: 10.11999/JEIT180485.
    [17] 王勇, 王沙沙, 田增山, 等. 基于FMCW雷达的双流融合神经网络手势识别方法[J]. 电子学报, 2019, 47(7): 1408–1415. doi: 10.3969/j.issn.0372-2112.2019.07.003.

    WANG Yong, WANG Shasha, TIAN Zengshan, et al. Two-stream fusion neural network approach for hand gesture recognition based on FMCW radar[J]. Acta Electronica Sinica, 2019, 47(7): 1408–1415. doi: 10.3969/j.issn.0372-2112.2019.07.003.
    [18] AHMED S, KIM W, PARK J, et al. Radar-based air-writing gesture recognition using a novel multistream CNN approach[J]. IEEE Internet of Things Journal, 2022, 9(23): 23869–23880. doi: 10.1109/JIOT.2022.3189395.
    [19] ALIREZAZAD K and MAURER L. FMCW radar-based hand gesture recognition using dual-stream CNN-GRU model[C]. 2022 24th International Microwave and Radar Conference, Gdansk, Poland, 2022: 1–5.
    [20] PARK G, CHANDRASEGAR V K, PARK J, et al. Increasing accuracy of hand gesture recognition using convolutional neural network[C]. 2022 International Conference on Artificial Intelligence in Information and Communication, Jeju Island, Korea, Republic of, 2022: 251–255.
    [21] DANG T L, NGUYEN H T, DAO D M, et al. SHAPE: A dataset for hand gesture recognition[J]. Neural Computing and Applications, 2022, 34(24): 21849–21862. doi: 10.1007/s00521-022-07651-1.
    [22] WANG Lingling, CHEN Xiaoyan, XIONG Wei, et al. Research on gesture recognition and classification based on attention mechanism[C]. 2022 IEEE 5th Advanced Information Management, Communicates, Electronic and Automation Control Conference, Chongqing, China, 2022: 1617–1621.
    [23] FANG Juan, XU Chao, WANG Chao, et al. Dynamic gesture recognition based on multimodal fusion model[C]. 2021 20th International Conference on Ubiquitous Computing and Communications, London, United Kingdom, 2021: 172–177.
    [24] GUO He, ZHANG Rui, LI Yang, et al. Research on human-vehicle gesture interaction technology based on computer visionbility[C]. 2022 IEEE 6th Advanced Information Technology, Electronic and Automation Control Conference, Beijing, China, 2022: 1161–1165.
    [25] STOVE A G. Linear FMCW radar techniques[J]. IEE Proceedings F (Radar and Signal Processing), 1992, 139(5): 343–350. doi: 10.1049/ip-f-2.1992.0048.
    [26] WINKLER V. Range Doppler detection for automotive FMCW radars[C]. 2007 European Microwave Conference, Munich, Germany, 2007: 166–169.
    [27] 夏朝阳, 周成龙, 介钧誉, 等. 基于多通道调频连续波毫米波雷达的微动手势识别[J]. 电子与信息学报, 2020, 42(1): 164–172. doi: 10.11999/JEIT190797.

    XIA Zhaoyang, ZHOU Chenglong, JIE Junyu, et al. Micro-motion gesture recognition based on multi-channel frequency modulated continuous wave millimeter wave radar[J]. Journal of Electronics &Information Technology, 2020, 42(1): 164–172. doi: 10.11999/JEIT190797.
    [28] 樊瑞宣, 姜高霞, 王文剑. 一种个性化k近邻的离群点检测算法[J]. 小型微型计算机系统, 2020, 41(4): 752–757. doi: 10.3969/j.issn.1000-1220.2020.04.014.

    FAN Ruixuan, JIANG Gaoxia, and WANG Wenjian. Outlier detection algorithm with personalized k-nearest neighbor[J]. Journal of Chinese Computer Systems, 2020, 41(4): 752–757. doi: 10.3969/j.issn.1000-1220.2020.04.014.
    [29] WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module[C]. The European Conference on Computer Vision, Munich, Germany, 2018: 3–19.
    [30] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 2261–2269.
  • 加载中
图(10) / 表(4)
计量
  • 文章访问数:  447
  • HTML全文浏览量:  200
  • PDF下载量:  155
  • 被引次数: 0
出版历程
  • 收稿日期:  2023-03-16
  • 修回日期:  2023-09-07
  • 网络出版日期:  2023-09-12
  • 刊出日期:  2024-03-27

目录

    /

    返回文章
    返回