Advanced Search
Volume 43 Issue 9
Sep.  2021
Turn off MathJax
Article Contents
Wenze SHAO, Miaomiao ZHANG, Haibo LI. Tiny Face Hallucination via Relativistic Adversarial Learning[J]. Journal of Electronics & Information Technology, 2021, 43(9): 2577-2585. doi: 10.11999/JEIT200362
Citation: Wenze SHAO, Miaomiao ZHANG, Haibo LI. Tiny Face Hallucination via Relativistic Adversarial Learning[J]. Journal of Electronics & Information Technology, 2021, 43(9): 2577-2585. doi: 10.11999/JEIT200362

Tiny Face Hallucination via Relativistic Adversarial Learning

doi: 10.11999/JEIT200362
Funds:  The Natural National Science Foundation of China (61771250, 61972213, 11901299), The Fundamental Research Funds for the Central Universities (30918014108)
  • Received Date: 2020-05-08
  • Rev Recd Date: 2020-10-18
  • Available Online: 2021-08-11
  • Publish Date: 2021-09-16
  • Considering that previous tiny face hallucination methods either produced visually less pleasant faces or required architecturally more complex networks, this paper advocates a new deep model for tiny face hallucination by borrowing the idea of Relativistic Generative Adversarial Network (tfh-RGAN). Specifically, a hallucination generator and a relativistic discriminator are jointly learned in an alternately iterative training fashion by minimizing the combined pixel loss and relativistic generative adversarial loss. As for the generator, it is mainly structured as concatenation of a few basic modules followed by three 2×up-sampling layers, and each basic module is formulated by coupling the residual blocks, dense blocks, and depthwise separable convolution operators. As such, the generator can be made lightweight while with a considerable depth so as to achieve high quality face hallucination. As for the discriminator, it makes use of VGG128 while removing all its batch normalization layers and embedding a fully connected layer additionally so as to fulfill the capacity limit of relativistic adversarial learning. Experimental results reveal that, the proposed method, though simpler in the network architecture without a need of explicitly imposing any face structural prior, is able to produce better hallucination faces with higher definition and stronger reality. In terms of the quantitative assessment, the peak signal-to-noise ratio of the proposed method can be improved up to 0.25~1.51 dB compared against several previous approaches.
  • loading
  • [1]
    DONG Chao, LOY C C, HE Kaiming, et al. Learning a deep convolutional network for image super-resolution[C]. Proceedings of the 13th European Conference on Computer Vision, Zurich, 2014: 184–199. doi: 10.1007/978-3-319-10593-2_13.
    [2]
    赵小强, 宋昭漾. 多级跳线连接的深度残差网络超分辨率重建[J]. 电子与信息学报, 2019, 41(10): 2501–2508. doi: 10.11999/JEIT190036

    ZHAO Xiaoqiang and SONG Zhaoyang. Super-resolution reconstruction of deep residual network with multi-level skip connections[J]. Journal of Electronics &Information Technology, 2019, 41(10): 2501–2508. doi: 10.11999/JEIT190036
    [3]
    GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]. Proceedings of the 27th International Conference on Neural Information Processing Systems, Cambridge, UK, 2014: 2672–2680.
    [4]
    YU Xin and PORIKLI F. Ultra-resolving face images by discriminative generative networks[C]. Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands, 2016: 318–333. doi: 10.1007/978-3-319-46454-1_20.
    [5]
    LUCIC M, KURACH K, MICHALSKI M, et al. Are GANs created equal? a large-scale study[EB/OL]. https://arxiv.org/abs/1711.10337, 2018.
    [6]
    SHAO Wenze, XU Jingjing, CHEN Long, et al. On potentials of regularized Wasserstein generative adversarial networks for realistic hallucination of tiny faces[J]. Neurocomputing, 2019, 364: 1–15. doi: 10.1016/j.neucom.2019.07.046
    [7]
    GULRAJANI I, AHMED F, ARJOVSKY M, et al. Improved training of Wasserstein GANS[EB/OL]. https://arxiv.org/abs/1704.00028, 2017.
    [8]
    CHEN Yu, TAI Ying, LIU Xiaoming, et al. FSRNet: End-to-end learning face super-resolution with facial priors[C]. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 2492–2501. doi: 10.1109/CVPR.2018.00264.
    [9]
    JOLICOEUR-MARTINEAU A. The relativistic discriminator: A key element missing from standard GAN[EB/OL]. https://arxiv.org/abs/1807.00734, 2018.
    [10]
    HE Kaiming, ZHANG Xiangyu, and REN Shaoqing, et al. Deep residual learning for image recognition[C]. Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. doi: 10.1109/CVPR.2016.90.
    [11]
    HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]. Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 2261–2269. doi: 10.1109/CVPR.2017.243.
    [12]
    HOWARD A G, ZHU Menglong, CHEN Bo, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications[EB/OL]. http://arxiv.org/abs/1704.04861, 2017.
    [13]
    WANG Xintao, YU Ke, WU Shixiang, et al. ESRGAN: Enhanced super-resolution generative adversarial networks[C]. Proceedings of the 2018 European Conference on Computer Vision, Munich, Germany, 2018: 63–79. doi: 10.1007/978-3-030-11021-5_5.
    [14]
    SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. Proceedings of the 3rd International Conference on Learning Representations, San Diego, USA, 2015: 1–14.
    [15]
    LIU Ziwei, LUO Ping, WANG Xiaogang, et al. Deep learning face attributes in the wild[C]. Proceedings of 2015 IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 3730–3738. doi: 10.1109/ICCV.2015.425.
    [16]
    KINGMA D P and BA J. Adam: A method for stochastic optimization[EB/OL]. https://arxiv.org/abs/1412.6980, 2017.
    [17]
    SONG Yibing, ZHANG Jiawei, HE Shengfeng, et al. Learning to hallucinate face images via component generation and enhancement[C]. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, Melbourne, Australia, 2017: 4537–4543.
    [18]
    JIANG Junjun, HU Yi, HU Jinhui, et al. Deep CNN denoiser and multi-layer neighbor component embedding for face hallucination[C]. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 2018: 771–778.
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(5)  / Tables(2)

    Article Metrics

    Article views (686) PDF downloads(64) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return