高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

一种用于细粒度人脸识别的眼镜去除方法

毛亮 薛月菊 魏颖慧 朱婷婷

毛亮, 薛月菊, 魏颖慧, 朱婷婷. 一种用于细粒度人脸识别的眼镜去除方法[J]. 电子与信息学报, 2021, 43(5): 1448-1456. doi: 10.11999/JEIT200176
引用本文: 毛亮, 薛月菊, 魏颖慧, 朱婷婷. 一种用于细粒度人脸识别的眼镜去除方法[J]. 电子与信息学报, 2021, 43(5): 1448-1456. doi: 10.11999/JEIT200176
Liang MAO, Yueju XUE, Yinghui WEI, Tingting ZHU. An Eyeglasses Removal Method for Fine-grained Face Recognition[J]. Journal of Electronics & Information Technology, 2021, 43(5): 1448-1456. doi: 10.11999/JEIT200176
Citation: Liang MAO, Yueju XUE, Yinghui WEI, Tingting ZHU. An Eyeglasses Removal Method for Fine-grained Face Recognition[J]. Journal of Electronics & Information Technology, 2021, 43(5): 1448-1456. doi: 10.11999/JEIT200176

一种用于细粒度人脸识别的眼镜去除方法

doi: 10.11999/JEIT200176
基金项目: 国家科技支撑计划(2015BAD06B03-3)
详细信息
    作者简介:

    毛亮:男,1983年生,副研究员,研究方向计算机视觉与深度学习

    薛月菊:女,1969年生,教授,主要研究方向机器视觉与图像处理计算机视觉与深度学习

    通讯作者:

    薛月菊 xueyueju@163.com

  • 中图分类号: TN911.73; TP391.4

An Eyeglasses Removal Method for Fine-grained Face Recognition

Funds: The National Science and Technology Support Program (2015BAD06B03-3)
  • 摘要: 为解决眼镜遮挡会降低人脸识别性能的难点,借鉴深度卷积神经网络在超分辨率方面的成功应用,该文提出一种用于细粒度人脸识别的眼镜自动去除方法ERCNN。用卷积层、池化层、MFM特征选取模块和反卷积层设计ERCNN网络模型,自动学习戴眼镜和未戴眼镜人脸图像对之间的映射关系,实现端到端的眼镜去除。然后,收集大量监控场景下的人脸图像,以及互联网上公开的人脸图像作为训练集;同时构建SLLFW数据集,作为眼镜去除和人脸识别的测试集。最后,通过与传统的眼镜去除方法进行对比试验,该文算法的各项评价指标优于传统方法,能有效的去除真实人脸图像中眼镜;同时在SLLFW人脸数据集上形成的全框眼镜、半框眼镜和无框眼镜人脸数据集上对多种人脸识别算法进行对比试验。试验表明,在FAR为1%的情况下,利用该文方法对F-SLLFW, H-SLLFW和R-SLLFW数据集的人脸图像进行眼镜去除后,SphereFace算法的TAR分别达到90.05%, 91.14%和92.33%,比未去除眼镜的识别率分别提高了3.92%, 3.08%和1.26%;同样,在FAR为0.1%的情况下,比SphereFace算法的TAR分别提高了10.06%, 4.29%和2.13%,说明该文方法有助于提升细粒度人脸识别的识别精度。
  • 图  1  戴不同类型眼镜的人脸图像

    图  2  人脸数据集

    图  3  ERCNN的网络结构

    Conv2a, ···, Conv2d, Conv4为卷积层;ReLU3为激活函数

    图  4  人脸眼镜去除的结果

    图  5  真实戴眼镜人脸的眼镜去除的结果

    图  6  真实戴眼镜人脸的眼镜去除效果不佳的结果

    表  1  不同方法以及不同类型的眼镜去除性能对比

    人脸对齐眼镜类型方法评价指标
    PSNR (dB)SSIMMSEIFC
    128×128全框ERCNN34.70.9822.037.0
    PCA24.20.85247.226.1
    半框ERCNN35.70.9917.57.3
    PCA25.50.87183.276.2
    无框ERCNN35.70.9917.57.5
    PCA26.00.88163.346.3
    平均值ERCNN35.30.9819.017.2
    PCA25.20.86197.946.2
    112×96全框ERCNN35.60.9917.917.3
    PCA24.30.84241.594.5
    半框ERCNN36.30.9915.247.6
    PCA25.00.85205.634.5
    无框ERCNN36.10.9915.967.7
    PCA24.90.85210.424.6
    平均值ERCNN36.00.9916.377.5
    PCA24.70.84219.214.5
    下载: 导出CSV

    表  2  不同人脸识别的识别性能对比(%)

    数据集FARTAR
    LFW199.0398.1399.0799.40
    0.195.5091.2795.9097.04
    SLLFW193.1778.4392.5396.11
    0.184.3362.2385.4391.38
    F-SLLFW180.8755.7783.2086.13
    0.168.2739.8770.7071.73
    本文方法对F-SLLFW进行处理后186.4764.4788.6390.05
    0.177.0748.1077.8781.79
    H-SLLFW183.1060.7086.2788.06
    0.172.8043.5375.1078.36
    本文方法对H-SLLFW进行处理后187.6067.0789.9391.14
    0.180.7045.5378.8382.65
    R-SLLFW187.0766.1089.3791.70
    0.179.5349.6776.7382.76
    本文方法对R-SLLFW进行处理后188.5367.5089.1392.33
    0.181.0747.8077.4384.89
    下载: 导出CSV
  • [1] 李小薪, 梁荣华. 有遮挡人脸识别综述: 从子空间回归到深度学习[J]. 计算机学报, 2018, 41(1): 177–207. doi: 10.11897/SP.J.1016.2018.00177

    LI Xiaoxin and LIANG Ronghua. A review for face recognition with occlusion: From subspace regression to deep learning[J]. Chinese Journal of Computers, 2018, 41(1): 177–207. doi: 10.11897/SP.J.1016.2018.00177
    [2] WU Chenyu, LIU Ce, SHUM H Y, et al. Automatic eyeglasses removal from face images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(3): 322–336. doi: 10.1109/TPAMI.2004.1262319
    [3] DU Cheng and SU Guangda. Eyeglasses removal from facial images[J]. Pattern Recognition Letters, 2005, 26(14): 2215–2220. doi: 10.1016/j.patrec.2005.04.002
    [4] PARK J S, OH Y H, AHN S C, et al. Glasses removal from facial image using recursive error compensation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(5): 805–811. doi: 10.1109/TPAMI.2005.103
    [5] HEO J and SAVVIDES M. Face pose correction with eyeglasses and occlusions removal[C]. 2007 IEEE Conference Biometrics Symposium, Baltimore, USA, 2007: 1–6. doi: 10.1109/BCC.2007.4430551.
    [6] JIA Xiaodong and GUO Jiangling. Eyeglasses removal from facial image based on phase congruency[C]. The 3rd IEEE International Congress on Image and Signal Processing, Yantai, China, 2010: 1859–1862, doi: 10.1109/CISP.2010.5647366.
    [7] WANG Yuankai, JANG J H, TSAI L W, et al. Improvement of face recognition by eyeglass removal[C]. The 6th IEEE International Conference on Intelligent Information Hiding and Multimedia Signal Processing, Darmstadt, Germany, 2010: 228–231. doi: 10.1109/IIHMSP.2010.64.
    [8] CHENG Wenchang, LIAO H C, PAN M H, et al. A fatigue detection system with eyeglasses removal[C]. The 15th IEEE International Conference on Advanced Communications Technology, PyeongChang, South Korea, 2013: 331–335.
    [9] 李根, 李文辉. 基于思维进化的机器学习的遮挡人脸识别[J]. 吉林大学学报: 工学版, 2014, 44(5): 1410–1416.

    LI Gen and LI Wenhui. Face occlusion recognition based on MEBML[J]. Journal of Jilin University:Engineering and Technology Edition, 2014, 44(5): 1410–1416.
    [10] GUO Pei and SU Fei. Enhanced PCA reconstruction method for eyeglass frame auto-removal[C]. The 4th IEEE International Conference on Network Infrastructure and Digital Content, Beijing, China, 2014: 359–363. doi: 10.1109/ICNIDC.2014.7000325.
    [11] 吴从中, 刘渠芬, 詹曙. 基于稀疏表达的人脸遮挡物去除[J]. 合肥工业大学学报: 自然科学版, 2015, 38(5): 615–617, 680.

    WU Congzhong, LIU Qufen, and ZHAN Shu. A face occlusion removal method based on sparse representation[J]. Journal of Hefei University of Technology, 2015, 38(5): 615–617, 680.
    [12] FERNÁNDEZ A, GARCÍA R, USAMENTIAGA R, et al. Glasses detection on real images based on robust alignment[J]. Machine Vision and Applications, 2015, 26(4): 519–531. doi: 10.1007/s00138-015-0674-1
    [13] SANDHAN T and CHOI J Y. Anti-glare: Tightly constrained optimization for eyeglass reflection removal[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 1675–1684. doi: 10.1109/CVPR.2017.182.
    [14] 曹志义, 牛少彰, 张继威. 基于半监督学习生成对抗网络的人脸还原算法研究[J]. 电子与信息学报, 2018, 40(2): 323–330. doi: 10.11999/JEIT170357

    CAO Zhiyi, NIU Shaozhang, ZHANG Jiwei. Research on face reduction algorithm based on generative adversarial nets with semi-supervised learning[J]. Journal of Electronics &Information Technology, 2018, 40(2): 323–330. doi: 10.11999/JEIT170357
    [15] 毛亮, 魏颖慧, 刘三阳, 等. 一种用于细粒度人脸识别的眼镜去除方法[P]. 中国专利, 110569763A, 2019.

    MAO Liang, WEI Yinghui, LIU Sanyang, et al. Glasses removing method for fine-grained face recognition[P]. China Patent, 110569763A, 2019.
    [16] DONG Chao, CHEN C L, HE Kaiming, et al. Learning a deep convolutional network for image super-resolution[C]. The 13th European Conference on Computer Vision, Zurich, Switzerland, 2014: 184–199. doi: 10.1007/978-3-319-10593-2_13.
    [17] WEN Yandong, ZHANG Kaipeng, LI Zhifeng, et al. A discriminative feature learning approach for deep face recognition[C]. The 14th European Conference on Computer Vision, Amsterdam, The Netherlands, 2016, 9911: 499–515. doi: 10.1007/978-3-319-46478-7_31.
    [18] WU Xiang, HE Ran, and SUN Zhenan. A lightened CNN for deep face representation[EB/OL]. https://arxiv.org/abs/1511.02683v1, 2015.
    [19] 谭小慧, 李昭伟, 樊亚春. 基于多尺度细节增强的面部表情识别方法[J]. 电子与信息学报, 2019, 41(11): 2752–2759. doi: 10.11999/EIT181088

    TAN Xiaohui, LI Zhaowei, and FAN Yachun. Facial expression recognition method based on multi-scale detail enhancement[J]. Journal of Electronics &Information Technology, 2019, 41(11): 2752–2759. doi: 10.11999/EIT181088
    [20] DENG Weihong, HU Jiani, ZHANG Nanhai, et al. Fine-grained face verification: FGLFW database, baselines, and human-DCMN partnership[J]. Pattern Recognition, 2017, 66: 63–73. doi: 10.1016/j.patcog.2016.11.023
    [21] SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. https://arxiv.org/abs/1409.1556v6, 2015.
    [22] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 1–9. doi: 10.1109/CVPR.2015.7298594.
    [23] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. doi: 10.1109/CVPR.2016.90.
    [24] JIA Yangqing, SHELHAMER E, DONAHUE J, et al. Caffe: Convolutional architecture for fast feature embedding[EB/OL]. https://arxiv.org/abs/1408.5093, 2014.
    [25] WANG Feng, CHENG Jian, LIU Weiyang, et al. Additive margin softmax for face verification[J]. IEEE Signal Processing Letters, 2018, 25(7): 926–930. doi: 10.1109/LSP.2018.2822810
    [26] DENG Jiankang, GUO Jia, XUE Niannan, et al. ArcFace: Additive angular margin loss for deep face recognition[EB/OL]. https://arxiv.org/abs/1801.07698, 2018.
    [27] LIU Weiyang, WEN Yandong, YU Zhiding, et al. SphereFace: Deep hypersphere embedding for face recognition[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 6738–6746. doi: 10.1109/CVPR.2017.713.
  • 加载中
图(6) / 表(2)
计量
  • 文章访问数:  2684
  • HTML全文浏览量:  1485
  • PDF下载量:  116
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-03-17
  • 修回日期:  2020-07-20
  • 网络出版日期:  2020-07-27
  • 刊出日期:  2021-05-18

目录

    /

    返回文章
    返回