高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于多尺度级联网络的水下图像增强方法

米泽田 晋洁 李圆圆 丁雪妍 梁政 付先平

米泽田, 晋洁, 李圆圆, 丁雪妍, 梁政, 付先平. 基于多尺度级联网络的水下图像增强方法[J]. 电子与信息学报, 2022, 44(10): 3353-3362. doi: 10.11999/JEIT220375
引用本文: 米泽田, 晋洁, 李圆圆, 丁雪妍, 梁政, 付先平. 基于多尺度级联网络的水下图像增强方法[J]. 电子与信息学报, 2022, 44(10): 3353-3362. doi: 10.11999/JEIT220375
MI Zetian, JIN Jie, LI Yuanyuan, DING Xueyan, LIANG Zheng, FU Xianping. Underwater Image Enhancement Method Based on Multi-scale Cascade Network[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3353-3362. doi: 10.11999/JEIT220375
Citation: MI Zetian, JIN Jie, LI Yuanyuan, DING Xueyan, LIANG Zheng, FU Xianping. Underwater Image Enhancement Method Based on Multi-scale Cascade Network[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3353-3362. doi: 10.11999/JEIT220375

基于多尺度级联网络的水下图像增强方法

doi: 10.11999/JEIT220375
基金项目: 国家自然科学基金(62176037),辽宁省重点研发计划(201801728)
详细信息
    作者简介:

    米泽田:女,副教授,研究方向为图像处理和机器学习等

    晋洁:女,硕士生,研究方向为图像处理等

    李圆圆:女,硕士生,研究方向为图像处理等

    丁雪妍:女,博士生,研究方向为水下图像增强等

    梁政:男,讲师,研究方向为计算机视觉和机器学习等

    付先平:男,教授,研究方向为图像处理和机器学习等

    通讯作者:

    米泽田 mizetian_group@163.com

  • 中图分类号: TN911.73

Underwater Image Enhancement Method Based on Multi-scale Cascade Network

Funds: The National Natural Science Foundation of China (62176037), The Foundation of Liaoning Province Key Research and Development Program (201801728)
  • 摘要: 针对水下图像由于光吸收、后向散射等因素导致的严重色偏、细节丢失等问题,该文提出一种基于多尺度级联网络的水下图像增强方法。针对单一网络特征利用不全面导致的图像梯度消失问题,该方法通过级联多尺度原始图像与相应的特征图像,以获得更优异的细节保持效果,并实现从较浅层到较深层快速预测残差的能力。此外,引入联合密集网络块和递归块,通过特征重用有效解决多尺度网络参数过多的问题。为有效解决单一损失造成的图像细节恢复不均的问题,提出Charbonnier和结构相似度 (SSIM) 联合损失函数。经仿真实验分析,所提网络在处理水下图像严重色偏、细节丢失等方面都取得了显著的效果。
  • 图  1  多尺度级联网络结构

    图  2  卷积块结构

    图  3  Recursive block结构

    图  4  UIEB数据集上不同方法的比较

    图  5  EUVP1数据集上不同方法的比较

    图  6  EUVP2数据集上不同方法的比较

    图  7  EUVP3数据集上不同方法的比较

    图  8  在UIEB数据集上其他情况的比较

    表  1  4种不同水下数据集的划分

    UIEBEUVP1EUVP2EUVP3
    训练图像对790499533301966
    测试图像对100555370219
    总图像对890555037002185
    下载: 导出CSV

    表  2  对网络框架结构的定量评价

    PSNRSSIMBlurPCQI
    A + C21.19800.69250.27420.9553
    A + B + C21.17220.70170.27020.9385
    下载: 导出CSV

    表  3  对多尺度子网络层数的定量评价

    PSNRSSIMBlurPCQI
    2层20.39550.67890.26530.9370
    3层(本文)21.17220.70170.27020.9385
    4层20.69140.69810.27770.9628
    5层19.64740.62900.27410.9220
    下载: 导出CSV

    表  4  对多尺度子网络中递归块的定量评价

    块数 PSNRSSIMBlurPCQI
    220.39710.67760.27330.9551
    3(本文)21.17220.70170.27020.9385
    421.42350.70000.27990.9435
    下载: 导出CSV

    表  5  对损失函数的定量评价

    PSNRSSIMBlurPCQI
    l17.13870.0028NaN0.1345
    Charbonnier21.28400.66870.27070.9194
    l1 + SSIM7.13870.0028NaN0.1345
    Charbonnier + SSIM(本文)21.17220.70170.27020.9385
    下载: 导出CSV

    表  6  在4种数据集上的PSNR定量评价(dB)

    UIEBEUVP1EUVP2EUVP3
    MIP17.782423.220819.035319.6164
    SMBLOT20.438519.915020.334419.2797
    UDCP12.570616.608518.629716.8542
    ULAP20.872923.880022.400622.8187
    RD20.926920.260519.284618.1743
    IBLA17.029623.786421.783322.7280
    UWCNN15.328523.288118.663120.0642
    UWGAN21.616313.341713.035310.5894
    Pix2pix23.014130.132525.124728.1732
    CycleGAN8.608122.05978.690525.1744
    本文21.172228.535423.362322.9755
    下载: 导出CSV

    表  7  在4种数据集上的SSIM定量评价

    UIEBEUVP1EUVP2EUVP3
    MIP0.48680.41550.48790.5424
    SMBLOT0.65710.49990.56370.6393
    UDCP0.47530.32820.45970.5401
    ULAP0.53730.49400.60540.7138
    RD0.64280.51160.53750.5079
    IBLA0.51880.44630.45850.6662
    UWCNN0.46600.57470.64210.6065
    UWGAN0.74280.03840.1218-0.0232
    Pix2pix0.65120.33900.38580.6748
    CycleGAN0.10840.48790.03180.7761
    本文0.70170.69760.74100.7307
    下载: 导出CSV

    表  8  在4种数据集上的PCQI定量评价

    UIEBEUVP1EUVP2EUVP3
    MIP0.69810.93580.66370.6033
    SMBLOT0.78301.01960.76970.6867
    UDCP0.56460.80610.59880.5058
    ULAP0.73590.99410.71590.6484
    RD0.84351.01760.76840.6724
    IBLA0.64530.96810.69110.6212
    UWCNN0.42770.76830.44450.4423
    UWGAN0.76860.04480.04990.1272
    Pix2pix0.53790.89150.54980.5996
    CycleGAN0.29570.50930.35570.5901
    本文0.93850.92090.74830.7753
    下载: 导出CSV

    表  9  在4种数据集上的Blur定量评价

    UIEBEUVP1EUVP2EUVP3
    MIP0.28400.25130.32280.3737
    SMBLOT0.28810.23970.33470.3778
    UDCP0.28440.24050.34240.3814
    ULAP0.28980.25180.33980.3858
    RD0.27460.24270.32520.3577
    IBLA0.28700.25250.34030.3884
    UWCNN0.29560.25180.34280.3876
    UWGAN0.33150.16640.17940.2880
    Pix2pix0.33190.26130.37690.4057
    CycleGAN0.31480.30860.35480.3305
    本文0.27020.26820.33620.2784
    下载: 导出CSV

    表  10  不同卷积神经网络模型参数量分析

    DenseNet121CycleGANUWCNNPix2pixUWGANMFFNUMUEN本文
    参数量(M)7.98107.950.1554.411.9311.9729.21.42
    下载: 导出CSV
  • [1] PIZER S M, AMBURN E P, AUSTIN J D, et al. Adaptive histogram equalization and its variations[J]. Computer Vision, Graphics, and Image Processing, 1987, 39(3): 355–368. doi: 10.1016/s0734-189x(87)80186-x
    [2] PIZER S M, JOHNSTON R E, ERICKSEN J P, et al. Contrast-limited adaptive histogram equalization: Speed and effectiveness[C]. The First Conference on Visualization in Biomedical Computing, Atlanta, USA, 1990: 337–345.
    [3] RIZZI A, GATTA C, and MARINI D. Color correction between gray world and white patch[J]. SPIE, 2002, 4662: 1–10.
    [4] VAN DE WEIJER J, GEVERS T, and GIJSENIJ A. Edge-based color constancy[J]. IEEE Transactions on Image Processing, 2007, 16(9): 2207–2214. doi: 10.1109/TIP.2007.901808
    [5] SINGH G, JAGGI N, VASAMSETTI S, et al. Underwater image/video enhancement using wavelet based color correction (WBCC) method[C]. 2015 IEEE Underwater Technology, Chennai, India, 2015: 1–5.
    [6] MUNIRAJ M and DHANDAPANI V. Underwater image enhancement by combining color constancy and dehazing based on depth estimation[J]. Neurocomputing, 2021, 460: 211–230. doi: 10.1016/j.neucom.2021.07.003
    [7] ZHUANG Peixian, LI Chongyi, and WU Jiamin. Bayesian retinex underwater image enhancement[J]. Engineering Applications of Artificial Intelligence, 2021, 101: 104171. doi: 10.1016/j.engappai.2021.104171
    [8] GALDRAN A, PARDO D, PICÓN A, et al. Automatic Red-Channel underwater image restoration[J]. Journal of Visual Communication and Image Representation, 2015, 26: 132–145. doi: 10.1016/j.jvcir.2014.11.006
    [9] LI Chongyi, GUO Jichang, CONG Runmin, et al. Underwater image enhancement by Dehazing with minimum information loss and histogram distribution prior[J]. IEEE Transactions on Image Processing, 2016, 25(12): 5664–5677. doi: 10.1109/TIP.2016.2612882
    [10] LI Chongyi, ANWAR S, and PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2020, 98: 107038. doi: 10.1016/j.patcog.2019.107038
    [11] WANG Nan, ZHOU Yabin, HAN Fenglei, et al. UWGAN: Underwater GAN for real-world underwater color restoration and Dehazing[J/OL]. https://arxiv.org/abs/1912.10269, 2019.
    [12] ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017: 2242–2251.
    [13] GUO Yecai, LI Hanyu, and ZHUANG Peixian. Underwater image enhancement using a multiscale dense generative adversarial network[J]. IEEE Journal of Oceanic Engineering, 2020, 45(3): 862–870. doi: 10.1109/JOE.2019.2911447
    [14] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]. The IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 2261–2269.
    [15] LAI Weisong, HUANG Jiabin, AHUJA N, et al. Fast and accurate image super-resolution with deep Laplacian pyramid networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(11): 2599–2613. doi: 10.1109/TPAMI.2018.2865304
    [16] WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.1109/TIP.2003.819861
    [17] LI Chongyi, GUO Chunle, REN Wenqi, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020, 29: 4376–4389. doi: 10.1109/TIP.2019.2955241
    [18] ISLAM M J, XIA Youya, and SATTAR J. Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3227–3234. doi: 10.1109/LRA.2020.2974710
    [19] WANG Shiqi, MA Kede, YEGANEH H, et al. A patch-structure representation method for quality assessment of contrast changed images[J]. IEEE Signal Processing Letters, 2015, 22(12): 2387–2390. doi: 10.1109/LSP.2015.2487369
    [20] CRETE F, DOLMIERE T, LADRET P, et al. The blur effect: Perception and estimation with a new no-reference perceptual blur metric[J]. SPIE, 2007, 6492, 64920I.
    [21] CARLEVARIS-BIANCO N, MOHAN A, and EUSTICE R M. Initial results in underwater single image dehazing[C]. OCEANS 2010 MTS/IEEE Seattle, Seattle, USA, 2010: 1–8.
    [22] SONG Wei, WANG Yan, HUANG Dongmei, et al. Enhancement of underwater images with statistical model of background light and optimization of transmission map[J]. IEEE Transactions on Broadcasting, 2020, 66(1): 153–169. doi: 10.1109/TBC.2019.2960942
    [23] DREWS P, DO NASCIMENTO E, MORAES F, et al. Transmission estimation in underwater single images[C]. 2013 IEEE International Conference on Computer Vision Workshops, Sydney, Australia, 2013: 825–830.
    [24] SONG Wei, WANG Yan, HUANG Dongmei, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]. 19th Pacific-Rim Conference on Advances in Multimedia Information Processing, Hefei, China, 2018: 678–688.
    [25] GHANI A S A and ISA N A. Underwater image quality enhancement through composition of dual-intensity images and Rayleigh-stretching[J]. SpringerPlus, 2014, 3: 757. doi: 10.1186/2193-1801-3-757
    [26] PENG Y T and COSMAN P C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579–1594. doi: 10.1109/TIP.2017.2663846
    [27] ISOLA P, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks[C]. The IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 5967–5976.
    [28] CHEN Renzhang, CAI Zhanchuan, and CAO Wei. MFFN: An underwater sensing scene image enhancement method based on multiscale feature fusion network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 4205612. doi: 10.1109/TGRS.2021.3134762
    [29] DING Xuan, DONG Hang, GUO Yu, et al. Uncertainly guided multi-scale underwater image enhancement network with tone mapping block[C]. 2021 China Automation Congress (CAC), Beijing, China, 2021: 7172–7176.
  • 加载中
图(8) / 表(10)
计量
  • 文章访问数:  918
  • HTML全文浏览量:  271
  • PDF下载量:  205
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-04-01
  • 修回日期:  2022-06-23
  • 网络出版日期:  2022-06-29
  • 刊出日期:  2022-10-19

目录

    /

    返回文章
    返回