高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于双重迭代的零样本低照度图像增强

向森 王应锋 邓慧萍 吴谨 喻莉

向森, 王应锋, 邓慧萍, 吴谨, 喻莉. 基于双重迭代的零样本低照度图像增强[J]. 电子与信息学报, 2022, 44(10): 3379-3388. doi: 10.11999/JEIT211593
引用本文: 向森, 王应锋, 邓慧萍, 吴谨, 喻莉. 基于双重迭代的零样本低照度图像增强[J]. 电子与信息学报, 2022, 44(10): 3379-3388. doi: 10.11999/JEIT211593
XIANG Sen, WANG Yingfeng, DENG Huiping, WU Jin, YU Li. Zero-shot Learning for Low-light Image Enhancement Based on Dual Iteration[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3379-3388. doi: 10.11999/JEIT211593
Citation: XIANG Sen, WANG Yingfeng, DENG Huiping, WU Jin, YU Li. Zero-shot Learning for Low-light Image Enhancement Based on Dual Iteration[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3379-3388. doi: 10.11999/JEIT211593

基于双重迭代的零样本低照度图像增强

doi: 10.11999/JEIT211593
基金项目: 国家自然科学基金(61702384, 62001180, 61871437)
详细信息
    作者简介:

    向森:男,副教授,研究方向为图像处理、机器学习

    王应锋:男,硕士生,研究方向为图像处理

    邓慧萍:女,副教授,研究方向为图像处理、机器学习

    吴谨:女,教授,研究方向为图像处理、机器学习

    喻莉:女,教授,研究方向为多媒体通信

    通讯作者:

    向森 xiangsen@wust.edu.cn

  • 1) 夜间实拍图像测试集网址: https://github.com/Feng-wy/Zero-shot-dual-iter-LLE
  • 中图分类号: TN911.73; TP391

Zero-shot Learning for Low-light Image Enhancement Based on Dual Iteration

Funds: The National Natural Science Foundation of China (61702384, 62001180, 61871437)
  • 摘要: 针对低光照条件下拍摄图像质量低下的问题,该文提出一种基于双重迭代的零样本低照度图像增强方法。其外层迭代通过卷积神经网络估计增强参数,再由内层迭代进行图像增强,增强结果进一步用于计算损失函数并反馈更新外层的参数估计网络,最终通过多轮迭代生成高质量的图像。在该框架下,还设计了多尺度增强系数估计模块、基于注意力的像素级大气光估计模块,并提出了基于亮度对比度、大气光、颜色均衡以及图像平滑性先验的无监督损失函数。大量实验结果表明,该方法可有效将低光照图像增强为高质量的清晰图像,其性能优于现有的同类方法。同时该方法基于零样本学习,不需任何训练数据集,具有良好的普适性。
  • 图  1  双重迭代示意图

    图  2  多尺度增强系数估计模块(MCEM)结构图

    图  3  基于注意力的大气光与全局大气光效果对比图

    图  4  基于注意力的大气光值估计模块(ATEM)结构图

    图  5  使用不同窗口大小的反相亮通道结果对比

    图  6  不同方法在LOL-v2测试集上的效果对比

    图  7  不同方法在真实夜间图像上的效果对比

    图  8  不同照度下的低照度图像和增强结果

    图  9  不同损失函数处理后的可视化结果

    图  10  双重迭代次数与SSIM指标的关系统计图

    表  1  双重迭代流程

     输入:低照度图像${J_0}$;初始权重${W_1}$;迭代次数$K$,$N$;学习率
        $\alpha $;迭代计数$n$=1,$k$;
     While($n < N$)do:
     载入权重${W_n}$;
     计算:$ D_n^{{\text{out}}}({J_0},{W_n}) $,$ A_l^n = {\text{MCEM}}{}_l({J_0}) $,${E^n} = {\text{ATEM}}({J_0})$;
        初始化$k$=1;
     While($k < K$)do:
     计算:$ J_k^n = D_k^{{\text{in}}}(J_{k - 1}^n,A_l^n,{E^n}) $;
     End While
     经过$K$次内层迭代,第$n$次外层迭代的中间结果为:$J_K^n$
     根据$J_K^n$计算损失函数${L_{{\text{total}}}}$和梯度$\dfrac{{\partial {L_{{\text{total}}}}}}{{\partial {W_n}}}$;
     更新权重:${W_{n + 1}} = {W_n} - \alpha \cdot \dfrac{{\partial {L_{{\text{total}}}}}}{{\partial {W_n}}}$;
     End While
     输出:最终结果为$J_K^N$
    下载: 导出CSV

    表  2  多尺度增强系数估计模块(MCEM)网络参数表

    模块参数下采样多尺度特征融合上采样
    MCEM(l=1)输出通道数32, 32, 6432, 32, 32, 3232, 32, 3
    激活函数ReLUReLU, ReLUReLU, ReLU, Tanh
    MCEM(l=2)输出通道数32,32,6432, 32, 32, 3232, 32, 3
    激活函数ReLUReLU, ReLUReLU, ReLU, Tanh
    MCEM(l=3)输出通道数32,32,6432, 32, 32, 3232, 32, 3
    激活函数ReLUReLU, ReLUReLU, ReLU, Tanh
    下载: 导出CSV

    表  3  LOL-v2测试集上不同方法的客观评价指标

    指标LIMERetinexNetEnlightenGANZero-DCERRDNet本文方法
    PSNR(dB)↑18.0616.2618.5218.9416.2320.68
    SSIM↑0.570.490.590.590.530.62
    MAE↓9.4510.387.788.3512.717.21
    VGG16-PFS↓1.071.661.040.981.530.91
    CEIQ↑2.993.073.162.912.643.15
    NIQE↓6.566.386.356.396.666.23
    SSEQ↓29.2437.1523.2226.0421.7126.58
    下载: 导出CSV

    表  4  夜间实拍图像测试集上不同方法的客观评价指标

    指标LIMERetinexNetEnlightenGANZero-DCERRDNet本文方法
    CEIQ↑2.892.943.022.772.463.01
    NIQE↓6.286.117.436.356.866.06
    SSEQ↓14.5827.1318.9014.3116.5412.73
    下载: 导出CSV

    表  5  消融实验结果

    指标$ \beta {L_{{\text{bc}}}} $$\beta {L_{{\text{bc}}}} + \gamma {L_{{\text{cwb}}}}$$\beta {L_{{\text{bc}}}} + \gamma {L_{{\text{cwb}}}} + \mu {L_{{\text{sm}}}}$${L_{{\text{total}}}}$
    PSNR(dB)13.6614.7618.1220.68
    SSIM0.400.480.570.62
    下载: 导出CSV
  • [1] 杨燕, 王志伟. 基于均值不等关系优化的自适应图像去雾算法[J]. 电子与信息学报, 2020, 42(3): 755–763. doi: 10.11999/JEIT190368

    YANG Yan and WANG Zhiwei. Adaptive image dehazing algorithm based on mean unequal relation optimization[J]. Journal of Electronics &Information Technology, 2020, 42(3): 755–763. doi: 10.11999/JEIT190368
    [2] PAN Jinshan, REN Wenqi, HU Zhe, et al. Learning to Deblur images with exemplars[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(6): 1412–1425. doi: 10.1109/TPAMI.2018.2832125
    [3] LI Chongyi, ANWAR S, and PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2020, 98: 107038. doi: 10.1016/j.patcog.2019.107038
    [4] 陈勇, 詹帝, 刘焕淋. 基于物理模型与边界约束的低照度图像增强算法[J]. 电子与信息学报, 2017, 39(12): 2962–2969. doi: 10.11999/JEIT170267

    CHEN Yong, ZHAN Di, and LIU Huanlin. Enhancement algorithm for low-lighting images based on physical model and boundary constraint[J]. Journal of Electronics &Information Technology, 2017, 39(12): 2962–2969. doi: 10.11999/JEIT170267
    [5] 柳长源, 王琪, 毕晓君. 基于多通道多尺度卷积神经网络的单幅图像去雨方法[J]. 电子与信息学报, 2020, 42(9): 2285–2292. doi: 10.11999/JEIT190755

    LIU Changyuan, WANG Qi, and BI Xiaojun. Research on rain removal method for single image based on multi-channel and multi-scale CNN[J]. Journal of Electronics &Information Technology, 2020, 42(9): 2285–2292. doi: 10.11999/JEIT190755
    [6] PIZER S M, AMBURN E P, AUSTIN J D, et al. Adaptive histogram equalization and its variations[J]. Computer Vision, Graphics, and Image Processing, 1987, 39(3): 355–368. doi: 10.1016/S0734-189X(87)80186-X
    [7] IBRAHIM H and KONG N S P. Brightness preserving dynamic histogram equalization for image contrast enhancement[J]. IEEE Transactions on Consumer Electronics, 2007, 53(4): 1752–1758. doi: 10.1109/TCE.2007.4429280
    [8] RAHMAN Z, JOBSON D J, and WOODELL G A. Multi-scale retinex for color image enhancement[C]. Proceedings of 3rd IEEE International Conference on Image Processing, Lausanne, Switzerland, 1996: 1003–1006.
    [9] GUO Xiaojie, LI Yu, and LING Haibin. LIME: Low-light image enhancement via illumination map estimation[J]. IEEE Transactions on Image Processing, 2017, 26(2): 982–993. doi: 10.1109/TIP.2016.2639450
    [10] LI Mading, LIU Jiaying, YANG Wenhan, et al. Structure-revealing low-light image enhancement via robust retinex model[J]. IEEE Transactions on Image Processing, 2018, 27(6): 2828–2841. doi: 10.1109/TIP.2018.2810539
    [11] DONG Xuan, WANG Guan, PANG Yi, et al. Fast efficient algorithm for enhancement of low lighting video[C]. 2011 IEEE International Conference on Multimedia and Expo, Barcelona, Spain, 2011: 1–6.
    [12] LORE K G, AKINTAYO A, SARKAR S, et al. LLNet: A deep autoencoder approach to natural low-light image enhancement[J]. Pattern Recognition, 2017, 61: 650–662. doi: 10.1016/j.patcog.2016.06.008
    [13] WEI Chen, WANG Wenjing, YANG Wenhan, et al. Deep Retinex decomposition for low-light enhancement[EB/OL]. https://arxiv.org/abs/1808.04560, 2018.
    [14] REN Wenqi, LIU Sifei, MA Lin, et al. Low-light image enhancement via a deep hybrid network[J]. IEEE Transactions on Image Processing, 2019, 28(9): 4364–4375. doi: 10.1109/TIP.2019.2910412
    [15] JIANG Yifan, GONG Xinyu, LIU Ding, et al. EnlightenGAN: Deep light enhancement without paired supervision[J]. IEEE Transactions on Image Processing, 2021, 30: 2340–2349. doi: 10.1109/TIP.2021.3051462
    [16] GUO Chunle, LI Chongyi, GUO Jichang, et al. Zero-reference deep curve estimation for low-light image enhancement[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, USA, 2020: 1777–1786.
    [17] ZHU Anqi, ZHANG Lin, SHEN Ying, et al. Zero-shot restoration of underexposed images via robust Retinex decomposition[C]. 2020 IEEE International Conference on Multimedia and Expo (ICME), London, UK, 2020: 1–6.
    [18] 方明, 刘小晗, 付飞蚺. 基于注意力的多尺度水下图像增强网络[J]. 电子与信息学报, 2021, 43(12): 3513–3521. doi: 10.11999/JEIT200836

    FANG Ming, LIU Xiaohan, and FU Feiran. Multi-scale underwater image enhancement network based on attention mechanism[J]. Journal of Electronics &Information Technology, 2021, 43(12): 3513–3521. doi: 10.11999/JEIT200836
    [19] LI Tao, ZHU Chuang, SONG Jiawen, et al. Low-light image enhancement using CNN and bright channel prior[C]. IEEE International Conference on Image Processing (ICIP), Beijing, China, 2017: 3215–3219,
    [20] HE Kaiming, SUN Jian, TANG Xiaoou, et al. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341–2353. doi: 10.1109/TPAMI.2010.168
    [21] LIU Ziwei, LI Xiaoxiao, LUO Ping, et al. Deep learning markov random field for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(8): 1814–1828. doi: 10.1109/TPAMI.2017.2737535
    [22] MERTENS T, KAUTZ J, and VAN REETH F. Exposure fusion: A simple and practical alternative to high dynamic range photography[J]. Computer Graphics Forum, 2009, 28(1): 161–171. doi: 10.1111/j.1467-8659.2008.01171.x
    [23] YANG Wenhan, WANG Wenjing, HUANG Haofeng, et al. Sparse gradient regularized deep retinex network for robust low-light image enhancement[J]. IEEE Transactions on Image Processing, 2021, 30: 2072–2086. doi: 10.1109/TIP.2021.3050850
    [24] SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].https://arxiv.org/abs/1409.1556, 2014.
    [25] MITTAL A, SOUNDARARAJAN R, and BOVIK A C. Making a 'Completely Blind' image quality analyzer[J]. IEEE Signal Processing Letters, 2013, 20(3): 209–212. doi: 10.1109/LSP.2012.2227726
    [26] LIU Lixiong, LIU Bao, HUANG Hua, et al. No-reference image quality assessment based on spatial and spectral entropies[J]. Signal Processing:Image Communication, 2014, 29(8): 856–863. doi: 10.1016/j.image.2014.06.006
    [27] WU Jun, XIA Zhaoqiang, REN Yifeng, et al. No-reference quality assessment for contrast-distorted image[C]. 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), Oulu, Finland, 2016: 1–5.
  • 加载中
图(10) / 表(5)
计量
  • 文章访问数:  1234
  • HTML全文浏览量:  414
  • PDF下载量:  231
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-12-29
  • 修回日期:  2022-03-19
  • 网络出版日期:  2022-04-17
  • 刊出日期:  2022-10-19

目录

    /

    返回文章
    返回