高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于调频连续波雷达的多维信息特征融合人体姿势识别方法

冯心欣 李文龙 何兆 郑海峰

冯心欣, 李文龙, 何兆, 郑海峰. 基于调频连续波雷达的多维信息特征融合人体姿势识别方法[J]. 电子与信息学报, 2022, 44(10): 3583-3591. doi: 10.11999/JEIT210696
引用本文: 冯心欣, 李文龙, 何兆, 郑海峰. 基于调频连续波雷达的多维信息特征融合人体姿势识别方法[J]. 电子与信息学报, 2022, 44(10): 3583-3591. doi: 10.11999/JEIT210696
FENG Xinxin, LI Wenlong, HE Zhao, ZHENG Haifeng. Human Posture Recognition Based on Multi-dimensional Information Feature Fusion of Frequency Modulated Continuous Wave Radar[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3583-3591. doi: 10.11999/JEIT210696
Citation: FENG Xinxin, LI Wenlong, HE Zhao, ZHENG Haifeng. Human Posture Recognition Based on Multi-dimensional Information Feature Fusion of Frequency Modulated Continuous Wave Radar[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3583-3591. doi: 10.11999/JEIT210696

基于调频连续波雷达的多维信息特征融合人体姿势识别方法

doi: 10.11999/JEIT210696
基金项目: 国家自然科学基金(61971139, 61601126),福建省自然科学基金(2021J01576)
详细信息
    作者简介:

    冯心欣:女,副教授,研究方向为智能物联网、车联网、张量理论及其应用、经济学理论及其应用等

    李文龙:男,硕士生,研究方向为深度学习、数据分析和边缘计算系统架构

    何兆:男,硕士生,研究方向为深度学习和雷达信号处理

    郑海峰:男,教授,博士生导师,研究方向为边缘计算、无线感知、机器学习、张量理论及其应用等

    通讯作者:

    郑海峰 zhenghf@fzu.edu.cn

  • 中图分类号: TN958

Human Posture Recognition Based on Multi-dimensional Information Feature Fusion of Frequency Modulated Continuous Wave Radar

Funds: The National Natural Science Foundation of China (61971139,61601126), The Natural Science Foundation of Fujian Province (2021J01576)
  • 摘要: 为实现在复杂多样的环境下人体姿势的识别,该文提出一种基于调频连续波(FMCW)雷达的多维信息特征融合的人体姿势识别方法。该方法通过对FMCW雷达原始信号进行3维快速傅里叶变换得到目标距离、速度和角度的多维信息,在采用具有噪声的基于密度的聚类算法(DBSCAN)和 Hampel滤波算法解决运动范围内动态或静态目标的噪声干扰后使用卷积神经网络对多维信息进行特征提取,然后利用低秩多模态融合网络(LMF)充分融合多维信息的特征,并通过域鉴别器进一步获得与环境无关的特征,最终使用活动识别器获得姿势识别结果。为了实用性,在边缘计算平台上搭载预先设计的算法和训练好的网络模型进行实验验证。实验结果表明,在复杂的环境下该方法的识别精度可达到91.5%。
  • 图  1  DV-EI-Net网络结构

    图  2  LMF结构图

    图  3  实验平台

    图  4  DBSCAN聚类效果

    图  5  Hampel滤波及线性插值效果

    图  6  4种姿势对应的数据集及识别结果

    表  1  DV-EI-Net单参数网络目标域分类精度(%)

    模型VTMDTM
    单参数网络(无域鉴别器)81.578.0
    单参数网络(有域鉴别器)84.580.0
    下载: 导出CSV

    表  2  姿势分类混淆矩阵(%)

    真实类别预测类别
    挥拳行走站立坐下
    挥拳91900
    行走59320
    站立30943
    坐下05788
    下载: 导出CSV

    表  3  DV-EI-Net多参数网络目标域分类精度(%)

    模型VTM+DTM
    基于串联特征融合方式(无域鉴别器)85.0
    基于LMF融合方式(无域鉴别器)86.5
    基于串联特征融合方式(有域鉴别器)87.5
    基于LMF融合方式(有域鉴别器)91.5
    下载: 导出CSV

    表  4  本文方法与其他方法平均精度的比较(%)

    模型数据集源域精度目标域精度
    CNN[21]单参数网络VTM86.582.0
    VGG16[22]单参数网络VTM87.083.5
    RDA-T[10]多参数网络VTM, DTM92.585.0
    CNN[21]单参数网络+域鉴别器VTM86.585.5
    VGG16[22]单参数网络+域鉴别器VTM87.586.5
    RDA-T[10]多参数网络+域鉴别器VTM, DTM92.588.5
    DV-EI-NetVTM, DTM94.091.5
    下载: 导出CSV
  • [1] 杨丽梅, 李致豪. 面向人机交互的手势识别系统设计[J]. 工业控制计算机, 2020, 33(3): 18–20,22. doi: 10.3969/j.issn.1001-182X.2020.03.007

    YANG Limei and LI Zhihao. Design of gesture recognition system towards human computer interaction[J]. Industrial Control Computer, 2020, 33(3): 18–20,22. doi: 10.3969/j.issn.1001-182X.2020.03.007
    [2] AGGARWAL J K and XIA Lu. Human activity recognition from 3D data: A review[J]. Pattern Recognition Letters, 2014, 48: 70–80. doi: 10.1016/j.patrec.2014.04.011
    [3] TRAN D, WANG Heng, TORRESANI L, et al. A closer look at spatiotemporal convolutions for action recognition[C]. The IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6450–6459.
    [4] 熊昕, 郑杨娇子, 张上. 基于长短时记忆网络及变体的跌倒检测和人体行为识别系统[J]. 信息通信, 2020(2): 65–67. doi: 10.3969/j.issn.1673-1131.2020.02.027

    XIONG Xin, ZHENG Yangjiaozi, and ZHANG Shang. Fall detection and human behavior recognition system based on long and short time memory networks and variants[J]. Information &Communications, 2020(2): 65–67. doi: 10.3969/j.issn.1673-1131.2020.02.027
    [5] SABOKROU M, POURREZA M, FAYYAZ M, et al. AVID: Adversarial visual irregularity detection[C]. 14th Asian Conference on Computer Vision, Perth, Australia, 2019: 488–505.
    [6] 刘天亮, 谯庆伟, 万俊伟, 等. 融合空间-时间双网络流和视觉注意的人体行为识别[J]. 电子与信息学报, 2018, 40(10): 2395–2401. doi: 10.11999/JEIT171116

    LIU Tianliang, QIAO Qingwei, WAN Junwei, et al. Human action recognition via Spatio-temporal dual network flow and visual attention fusion[J]. Journal of Electronics &Information Technology, 2018, 40(10): 2395–2401. doi: 10.11999/JEIT171116
    [7] WANG Jie, ZHANG Xiao, GAO Qinhua, et al. Device-free wireless localization and activity recognition: A deep learning approach[J]. IEEE Transactions on Vehicular Technology, 2017, 66(7): 6258–6267. doi: 10.1109/TVT.2016.2635161
    [8] XU Shengzhi, KOOIJ B J, and YAROVOY A. Joint Doppler and DOA estimation using (Ultra-)Wideband FMCW signals[J]. Signal Processing, 2020, 168: 107259. doi: 10.1016/j.sigpro.2019.107259
    [9] LEE J, HWANG S, YOU S, et al. Joint angle, velocity, and range estimation using 2D MUSIC and successive interference cancellation in FMCW MIMO radar system[J]. IEICE Transactions on Communications, 2020, E103.B(3): 283–290. doi: 10.1587/transcom.2018EBP3330
    [10] 王勇, 吴金君, 田增山, 等. 基于FMCW雷达的多维参数手势识别算法[J]. 电子与信息学报, 2019, 41(4): 822–829. doi: 10.11999/JEIT180485

    WANG Yong, WU Jinjun, TIAN Zengshan, et al. Gesture recognition with multi-dimensional parameter using FMCW radar[J]. Journal of Electronics &Information Technology, 2019, 41(4): 822–829. doi: 10.11999/JEIT180485
    [11] ZHAO Yinan, ZHANG Zihao, and ZHANG Zhaolin. Multi-angle data cube action recognition based on millimeter wave radar[C]. 2020 Chinese Control and Decision Conference (CCDC), Hefei, China, 2020: 749–753.
    [12] ZHAO Mingmin, LI Tianhong, ABU ALSHEIKH M, et al. Through-wall human pose estimation using radio signals[C]. The IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 7356–7365.
    [13] 刘皓, 郭立, 易波, 等. 基于3D骨架和MCRF模型的行为识别[J]. 中国科学技术大学学报, 2014, 44(4): 285–291. doi: 10.3969/j.issn.0253-2778.2014.04.005

    LIU Hao, GUO Li, YI Bo, et al. Human activity recognition based on 3D skeletons and MCRF model[J]. Journal of University of Science and Technology of China, 2014, 44(4): 285–291. doi: 10.3969/j.issn.0253-2778.2014.04.005
    [14] ATREY P K, HOSSAIN M A, EL SADDIK A, et al. Multimodal fusion for multimedia analysis: A survey[J]. Multimedia Systems, 2010, 16(6): 345–379. doi: 10.1007/s00530-010-0182-0
    [15] MORENCY L P, MIHALCEA R, and DOSHI P. Towards multimodal sentiment analysis: Harvesting opinions from the web[C]. The 13th International Conference on Multimodal Interfaces, Alicante, Spain, 2011: 169–176.
    [16] XUE Hongfei, JIANG Wenjun, MIAO Chenglin, et al. DeepFusion: A deep learning framework for the fusion of heterogeneous sensory data[C]. The Twentieth ACM International Symposium on Mobile Ad Hoc Networking and Computing, Catania, Italy, 2019: 151–160.
    [17] ZADEH A, CHEN Minghai, PORIA S, et al. Tensor fusion network for multimodal sentiment analysis[C]. The 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, 2017: 1114–1125.
    [18] LIU Zhun, SHEN Ying, LAKSHMINARASIMHAN V B, et al. Efficient low-rank multimodal fusion with modality-specific factors[C]. The 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, 2018: 2247–2256.
    [19] GANIN Y and LEMPITSKY V. Unsupervised domain adaptation by backpropagation[C]. The 32nd International Conference on Machine Learning, Lille, France, 2015: 1180–1189.
    [20] SCITOVSKI R, MAJSTOROVIĆ S, and SABO K. A combination of RANSAC and DBSCAN methods for solving the multiple geometrical object detection problem[J]. Journal of Global Optimization, 2021, 79(3): 669–686. doi: 10.1007/s10898-020-00950-8
    [21] DEKKER B, JACOBS S, KOSSEN A S, et al. Gesture recognition with a low power FMCW radar and a deep convolutional neural network[C]. 2017 European Radar Conference (EURAD), Nuremberg, Germany, 2017: 163–166.
    [22] SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. 3rd International Conference on Learning Representations, San Diego, USA, 2014.
  • 加载中
图(6) / 表(4)
计量
  • 文章访问数:  1039
  • HTML全文浏览量:  610
  • PDF下载量:  163
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-07-12
  • 修回日期:  2021-11-18
  • 录用日期:  2021-11-25
  • 网络出版日期:  2021-11-26
  • 刊出日期:  2022-10-19

目录

    /

    返回文章
    返回