An Eyeglasses Removal Method for Fine-grained Face Recognition

Liang MAO; Yueju XUE; Yinghui WEI; Tingting ZHU

doi:10.11999/JEIT200176

Volume 43 Issue 5

May 2021

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2021 > 43(5): 1448-1456

Liang MAO, Yueju XUE, Yinghui WEI, Tingting ZHU. An Eyeglasses Removal Method for Fine-grained Face Recognition[J]. Journal of Electronics & Information Technology, 2021, 43(5): 1448-1456. doi: 10.11999/JEIT200176

Citation:

Liang MAO, Yueju XUE, Yinghui WEI, Tingting ZHU. An Eyeglasses Removal Method for Fine-grained Face Recognition[J]. Journal of Electronics & Information Technology, 2021, 43(5): 1448-1456. doi: 10.11999/JEIT200176

Citation:

PDF( 2452 KB)

An Eyeglasses Removal Method for Fine-grained Face Recognition

doi: 10.11999/JEIT200176 cstr: 32379.14.JEIT200176

1.
Institute of Applied Artificial Intelligence of the Guangdong-Hong Kong-Macao Greater Bay Area, Shenzhen Polytechnic, Shenzhen 518055, China
2.
College of Electronic Engineering, South China Agricultural University, Guangzhou 510642, China
3.
School of Mathematics and Statistics, Xidian University, Xi’an 710071, China

Funds: The National Science and Technology Support Program (2015BAD06B03-3)

Received Date: 2020-03-17
Rev Recd Date: 2020-07-20

Available Online: 2020-07-27

Publish Date: 2021-05-18

Abstract

Abstract

In order to solve the problem that eyeglasses reduce often the performance of face recognition, based on the successful application of deep convolution neural network in super-resolution, This paper proposes an automatic eyeglasses removal method ERCNN (Eyeglasses Removal CNN) for fine-grained face recognition. Specifically, the ERCNN network which is designed based on the convolution layer, pool layer, MFM (Max Feature Map)feature selection module and deconvolution layer, are automatically learned the mapping relationship between facial images with eyeglasses and their counterparts without eyeglasses to realize end-to-end eyeglasses removal. Then, massive facial images are captured through surveillance equipment and collected from the Internet as the training set. And, SLLFW data set is established, which is used as the test set of eyeglasses removal and face recognition. The experiment show that the proposed method can better effectively remove the eyeglasses from the real facial image than the traditional eyeglasses removal methods, and the evaluation index of the method is better than other methods. In addition, several face recognition methods are tested separately on the facial images formed by SLLFW data set. Experiments show that when the FAR (False Accept Rate) is 1%, the TAR (True Accept Rate) of the Sphereface method reaches 90.05%, 91.14% and 92.33%, which is 3.92%, 3.08% and 1.26% higher than the Sphereface method is not used to remove the eyeglasses from the F-SLLFW, H-SLLF and R-SLLFW, respectively. Similarly, when the FAR 0.1%, the TAR of Sphereface method is increased by 10.06%, 3.08% and 1.26% respectively. Therefore, the proposed method can better improve the recognition accuracy of fine-grained face recognition.
- Face recognition,
- Deep Convolution Neural Network (DCNN),
- Feature selection,
- Eyeglasses removal

FullText(HTML)

References(27)

References

[1]	李小薪, 梁荣华. 有遮挡人脸识别综述: 从子空间回归到深度学习[J]. 计算机学报, 2018, 41(1): 177–207. doi: 10.11897/SP.J.1016.2018.00177 LI Xiaoxin and LIANG Ronghua. A review for face recognition with occlusion: From subspace regression to deep learning[J]. Chinese Journal of Computers, 2018, 41(1): 177–207. doi: 10.11897/SP.J.1016.2018.00177
[2]	WU Chenyu, LIU Ce, SHUM H Y, et al. Automatic eyeglasses removal from face images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(3): 322–336. doi: 10.1109/TPAMI.2004.1262319
[3]	DU Cheng and SU Guangda. Eyeglasses removal from facial images[J]. Pattern Recognition Letters, 2005, 26(14): 2215–2220. doi: 10.1016/j.patrec.2005.04.002
[4]	PARK J S, OH Y H, AHN S C, et al. Glasses removal from facial image using recursive error compensation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(5): 805–811. doi: 10.1109/TPAMI.2005.103
[5]	HEO J and SAVVIDES M. Face pose correction with eyeglasses and occlusions removal[C]. 2007 IEEE Conference Biometrics Symposium, Baltimore, USA, 2007: 1–6. doi: 10.1109/BCC.2007.4430551.
[6]	JIA Xiaodong and GUO Jiangling. Eyeglasses removal from facial image based on phase congruency[C]. The 3rd IEEE International Congress on Image and Signal Processing, Yantai, China, 2010: 1859–1862, doi: 10.1109/CISP.2010.5647366.
[7]	WANG Yuankai, JANG J H, TSAI L W, et al. Improvement of face recognition by eyeglass removal[C]. The 6th IEEE International Conference on Intelligent Information Hiding and Multimedia Signal Processing, Darmstadt, Germany, 2010: 228–231. doi: 10.1109/IIHMSP.2010.64.
[8]	CHENG Wenchang, LIAO H C, PAN M H, et al. A fatigue detection system with eyeglasses removal[C]. The 15th IEEE International Conference on Advanced Communications Technology, PyeongChang, South Korea, 2013: 331–335.
[9]	李根, 李文辉. 基于思维进化的机器学习的遮挡人脸识别[J]. 吉林大学学报: 工学版, 2014, 44(5): 1410–1416. LI Gen and LI Wenhui. Face occlusion recognition based on MEBML[J]. Journal of Jilin University:Engineering and Technology Edition, 2014, 44(5): 1410–1416.
[10]	GUO Pei and SU Fei. Enhanced PCA reconstruction method for eyeglass frame auto-removal[C]. The 4th IEEE International Conference on Network Infrastructure and Digital Content, Beijing, China, 2014: 359–363. doi: 10.1109/ICNIDC.2014.7000325.
[11]	吴从中, 刘渠芬, 詹曙. 基于稀疏表达的人脸遮挡物去除[J]. 合肥工业大学学报: 自然科学版, 2015, 38(5): 615–617, 680. WU Congzhong, LIU Qufen, and ZHAN Shu. A face occlusion removal method based on sparse representation[J]. Journal of Hefei University of Technology, 2015, 38(5): 615–617, 680.
[12]	FERNÁNDEZ A, GARCÍA R, USAMENTIAGA R, et al. Glasses detection on real images based on robust alignment[J]. Machine Vision and Applications, 2015, 26(4): 519–531. doi: 10.1007/s00138-015-0674-1
[13]	SANDHAN T and CHOI J Y. Anti-glare: Tightly constrained optimization for eyeglass reflection removal[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 1675–1684. doi: 10.1109/CVPR.2017.182.
[14]	曹志义, 牛少彰, 张继威. 基于半监督学习生成对抗网络的人脸还原算法研究[J]. 电子与信息学报, 2018, 40(2): 323–330. doi: 10.11999/JEIT170357 CAO Zhiyi, NIU Shaozhang, ZHANG Jiwei. Research on face reduction algorithm based on generative adversarial nets with semi-supervised learning[J]. Journal of Electronics &Information Technology, 2018, 40(2): 323–330. doi: 10.11999/JEIT170357
[15]	毛亮, 魏颖慧, 刘三阳, 等. 一种用于细粒度人脸识别的眼镜去除方法[P]. 中国专利, 110569763A, 2019. MAO Liang, WEI Yinghui, LIU Sanyang, et al. Glasses removing method for fine-grained face recognition[P]. China Patent, 110569763A, 2019.
[16]	DONG Chao, CHEN C L, HE Kaiming, et al. Learning a deep convolutional network for image super-resolution[C]. The 13th European Conference on Computer Vision, Zurich, Switzerland, 2014: 184–199. doi: 10.1007/978-3-319-10593-2_13.
[17]	WEN Yandong, ZHANG Kaipeng, LI Zhifeng, et al. A discriminative feature learning approach for deep face recognition[C]. The 14th European Conference on Computer Vision, Amsterdam, The Netherlands, 2016, 9911: 499–515. doi: 10.1007/978-3-319-46478-7_31.
[18]	WU Xiang, HE Ran, and SUN Zhenan. A lightened CNN for deep face representation[EB/OL]. https://arxiv.org/abs/1511.02683v1, 2015.
[19]	谭小慧, 李昭伟, 樊亚春. 基于多尺度细节增强的面部表情识别方法[J]. 电子与信息学报, 2019, 41(11): 2752–2759. doi: 10.11999/EIT181088 TAN Xiaohui, LI Zhaowei, and FAN Yachun. Facial expression recognition method based on multi-scale detail enhancement[J]. Journal of Electronics &Information Technology, 2019, 41(11): 2752–2759. doi: 10.11999/EIT181088
[20]	DENG Weihong, HU Jiani, ZHANG Nanhai, et al. Fine-grained face verification: FGLFW database, baselines, and human-DCMN partnership[J]. Pattern Recognition, 2017, 66: 63–73. doi: 10.1016/j.patcog.2016.11.023
[21]	SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. https://arxiv.org/abs/1409.1556v6, 2015.
[22]	SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 1–9. doi: 10.1109/CVPR.2015.7298594.
[23]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. doi: 10.1109/CVPR.2016.90.
[24]	JIA Yangqing, SHELHAMER E, DONAHUE J, et al. Caffe: Convolutional architecture for fast feature embedding[EB/OL]. https://arxiv.org/abs/1408.5093, 2014.
[25]	WANG Feng, CHENG Jian, LIU Weiyang, et al. Additive margin softmax for face verification[J]. IEEE Signal Processing Letters, 2018, 25(7): 926–930. doi: 10.1109/LSP.2018.2822810
[26]	DENG Jiankang, GUO Jia, XUE Niannan, et al. ArcFace: Additive angular margin loss for deep face recognition[EB/OL]. https://arxiv.org/abs/1801.07698, 2018.
[27]	LIU Weiyang, WEN Yandong, YU Zhiding, et al. SphereFace: Deep hypersphere embedding for face recognition[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 6738–6746. doi: 10.1109/CVPR.2017.713.