A Dual-stream Network Based on Body Contour Deformation Field for Gait Recognition

HUO Wei; WANG Ke; TANG Jun; WANG Nian; LIANG Dong

doi:10.11999/JEIT231025

Volume 46 Issue 10

Oct. 2024

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2024 > 46(10): 4062-4071

HUO Wei, WANG Ke, TANG Jun, WANG Nian, LIANG Dong. A Dual-stream Network Based on Body Contour Deformation Field for Gait Recognition[J]. Journal of Electronics & Information Technology, 2024, 46(10): 4062-4071. doi: 10.11999/JEIT231025

Citation:

HUO Wei, WANG Ke, TANG Jun, WANG Nian, LIANG Dong. A Dual-stream Network Based on Body Contour Deformation Field for Gait Recognition[J]. Journal of Electronics & Information Technology, 2024, 46(10): 4062-4071. doi: 10.11999/JEIT231025

Citation:

PDF( 3803 KB)

A Dual-stream Network Based on Body Contour Deformation Field for Gait Recognition

doi: 10.11999/JEIT231025 cstr: 32379.14.JEIT231025

HUO Wei¹,
WANG Ke²,
TANG Jun^{1
,
,},
WANG Nian¹,
LIANG Dong²

1.
School of Electronic and Information Engineering, Anhui University, Hefei 230601, China
2.
School of Internet, Anhui University, Hefei 230039, China

Funds: The National Natural Science Foundation of China (62273001, 61772032), Anhui Provincial Key Research and Development Project (2022k07020006), The Natural Science Research Key Project of Anhui Educational Committee (KJ2021ZD0004)

Received Date: 2023-09-19
Rev Recd Date: 2024-09-04

Available Online: 2024-09-16

Publish Date: 2024-10-30

Abstract

Abstract

Gait recognition is susceptible to external factors such as camera viewpoints, clothing, and carrying conditions, which could lead to performance degradation. To address these issues, the technique of non-rigid point set registration is introduced into gait recognition, which is used to improve the dynamic perception ability of human morphological changes by utilizing the deformation field between adjacent gait frames to represent the displacement of human contours during walking. Accordingly, a dual-flow convolutional neural network-GaitDef exploiting human contour deformation field is proposed in this paper, which consists of deformation field and gait silhouette extraction branches. Besides, a multi-scale feature extraction module is designed for the sparsity of deformation field data to obtain multi-level spatial structure information of the deformation field. A dynamic difference capture module and a context information augmentation module are proposed to capture the changing characteristics of dynamic regions in gait silhouettes and consequently enhance gait representation ability by utilizing context information. The output features of the dual-branch network structure are fused to obtain the final gait representation. Extensive experimental results verify the effectiveness of GaitDef. The average Rank-1 accuracy of GaitDef can achieve 93.5%和68.3% on CASIA-B and CCPG datasets, respectively.
- Gait recognition,
- Point set registration,
- Convolutional Neural Network (CNN),
- Feature fusion

FullText(HTML)

References(28)

References

[1]	杨旗, 薛定宇. 基于双尺度动态贝叶斯网络及多信息融合的步态识别[J]. 电子与信息学报, 2012, 34(5): 1148–1153. doi: 10.3724/SP.J.1146.2011.01012. YANG Qi and XUE Dingyu. Gait recognition based on two-scale dynamic Bayesian network and more information fusion[J]. Journal of Electronics & Information Technology, 2012, 34(5): 1148–1153. doi: 10.3724/SP.J.1146.2011.01012.
[2]	王茜, 蔡竞, 郭柏冬, 等. 面向公共安全的步态识别技术研究[J]. 中国人民公安大学学报: 自然科学版, 2023, 29(1): 68–76. doi: 10.3969/j.issn.1007-1784.2023.01.009. WANG Qian, CAI Jing, GUO Baidong, et al. Research on gait recognition technology for public security[J]. Journal of People’s Public Security University of China: Science and Technology, 2023, 29(1): 68–76. doi: 10.3969/j.issn.1007-1784.2023.01.009.
[3]	LIAO Rijun, YU Shiqi, AN Weizhi, et al. A model-based gait recognition method with body pose and human prior knowledge[J]. Pattern Recognition, 2020, 98: 107069. doi: 10.1016/j.patcog.2019.107069.
[4]	CAO Zhe, SIMON T, WEI S E, et al. Realtime multi-person 2D pose estimation using part affinity fields[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 1302–1310. doi: 10.1109/CVPR.2017.143.
[5]	TEEPE T, GILG J, HERZOG F, et al. Towards a deeper understanding of skeleton-based gait recognition[C]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 1568–1576. doi: 10.1109/CVPRW56347.2022.00163.
[6]	AN Weizhi, YU Shiqi, MAKIHARA Y, et al. Performance evaluation of model-based gait on multi-view very large population database with pose sequences[J]. IEEE Transactions on Biometrics, Behavior, and Identity Science, 2020, 2(4): 421–430. doi: 10.1109/tbiom.2020.3008862.
[7]	WANG Likai, CHEN Jinyan, and LIU Yuxin. Frame-level refinement networks for skeleton-based gait recognition[J]. Computer Vision and Image Understanding, 2022, 222: 103500. doi: 10.1016/j.cviu.2022.103500.
[8]	TEEPE T, KHAN A, GILG J, et al. Gaitgraph: Graph convolutional network for skeleton-based gait recognition[C]. 2021 IEEE International Conference on Image Processing, Anchorage, USA, 2021: 2314–2318. doi: 10.1109/icip42928.2021.9506717.
[9]	LIAO Rijun, CAO Chunshui, GARCIA E B, et al. Pose-based temporal-spatial network (PTSN) for gait recognition with carrying and clothing variations[C]. The 12th Chinese Conference on Biometric Recognition, Shenzhen, China, 2017: 474–483. doi: 10.1007/978-3-319-69923-3_51.
[10]	LI Xiang, MAKIHARA Y, XU Chi, et al. End-to-end model-based gait recognition[C]. The 15th Asian Conference on Computer Vision, Kyoto, Japan, 2020: 3–20. doi: 10.1007/978-3-030-69535-4_1.
[11]	HAN Ju and BHANU B. Individual recognition using gait energy image[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(2): 316–322. doi: 10.1109/TPAMI.2006.38.
[12]	LIU Jianyi and ZHENG Nanning. Gait history image: A novel temporal template for gait recognition[C]. 2007 IEEE International Conference on Multimedia and Expo, Beijing, China, 2007: 663–666. doi: 10.1109/ICME.2007.4284737.
[13]	CHEN Changhong, LIANG Jimin, ZHAO Heng, et al. Frame difference energy image for gait recognition with incomplete silhouettes[J]. Pattern Recognition Letters, 2009, 30(11): 977–984. doi: 10.1016/j.patrec.2009.04.012.
[14]	CHAO Hanqing, HE Yiwei, ZHANG Junping, et al. GaitSet: Regarding gait as a set for cross-view gait recognition[C]. The 33th AAAI Conference on Artificial Intelligence, Honolulu, USA, 2019: 8126–8133. doi: 10.1609/aaai.v33i01.33018126.
[15]	HOU Saihui, CAO Chunshui, LIU Xu, et al. Gait lateral network: Learning discriminative and compact representations for gait recognition[C]. The 16th European Conference on Computer Vision, Glasgow, UK, 2020: 382–398. doi: 10.1007/978-3-030-58545-7_22.
[16]	LIN Beibei, ZHANG Shunli, and YU Xin. Gait recognition via effective global-local feature representation and local temporal aggregation[C]. 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 2021: 14628–14636. doi: 10.1109/ICCV48922.2021.01438.
[17]	WANG Ming, LIN Beibei, GUO Xianda, et al. GaitStrip: Gait recognition via effective strip-based feature representations and multi-level framework[C]. The 16th Asian Conference on Computer Vision, Macao, China, 2023: 711–727. doi: 10.1007/978-3-031-26316-3_42.
[18]	LI Huakang, QIU Yidan, ZHAO Huimin, et al. GaitSlice: A gait recognition model based on spatio-temporal slice features[J]. Pattern Recognition, 2022, 124: 108453. doi: 10.1016/j.patcog.2021.108453.
[19]	FAN Chao, PENG Yunjie, CAO Chunhui, et al. GaitPart: Temporal part-based model for gait recognition[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 14213–14221. doi: 10.1109/CVPR42600.2020.01423.
[20]	HUANG Xiaohu, ZHU Duowang, WANG Hao, et al. Context-sensitive temporal feature learning for gait recognition[C]. 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 2021: 12889–12898. doi: 10.1109/ICCV48922.2021.01267.
[21]	LIN Beibei, ZHANG Shunli, and BAO Feng. Gait recognition with multiple-temporal-scale 3D convolutional neural network[C]. The 28th ACM International Conference on Multimedia, Seattle, USA, 2020: 3054–3062. doi: 10.1145/3394171.3413861.
[22]	CHAI Tianrui, LI Annan, ZHANG Shaoxiong, et al. Lagrange motion analysis and view embeddings for improved gait recognition[C]. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 20217–20226. doi: 10.1109/CVPR52688.2022.01961.
[23]	LIANG Junhao, FAN Chao, HOU Saihui, et al. GaitEdge: Beyond plain end-to-end gait recognition for better practicality[C]. The 17th European Conference on Computer Vision, Tel Aviv, Israel, 2022: 375–390. doi: 10.1007/978-3-031-20065-6_22.
[24]	CHUI H and RANGARAJAN A. A new point matching algorithm for non-rigid registration[J]. Computer Vision and Image Understanding, 2003, 89(2/3): 114–141. doi: 10.1016/s1077-3142(03)00009-2.
[25]	YU Shiqi, TAN Daoliang, and TAN Tieniu. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition[C]. The 18th International Conference on Pattern Recognition, Hong Kong, China 2006: 441–444. doi: 10.1109/icpr.2006.67.
[26]	LI Weijia, HOU Saihui, ZHANG Chunjie, et al. An in-depth exploration of person re-identification and gait recognition in cloth-changing conditions[C]. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, 2023: 13824–13833. doi: 10.1109/CVPR52729.2023.01328.
[27]	DOU Huanzhang, ZHANG Pengyi, SU Wei, et al. MetaGait: Learning to learn an Omni sample adaptive representation for gait recognition[C]. The 17th European Conference on Computer Vision, Tel Aviv, Israel, 2022: 357–374. doi: 10.1007/978-3-031-20065-6_21.
[28]	DOU Huanzhang, ZHANG Pengyi, SU Wei, et al. GaitGCI: Generative counterfactual intervention for gait recognition[C]. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, 2023: 5578–5588. doi: 10.1109/cvpr52729.2023.00540.