高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于特征解耦的无监督水下图像增强

刘彦呈 董张伟 朱鹏莅 刘厶源

刘彦呈, 董张伟, 朱鹏莅, 刘厶源. 基于特征解耦的无监督水下图像增强[J]. 电子与信息学报, 2022, 44(10): 3389-3398. doi: 10.11999/JEIT211517
引用本文: 刘彦呈, 董张伟, 朱鹏莅, 刘厶源. 基于特征解耦的无监督水下图像增强[J]. 电子与信息学报, 2022, 44(10): 3389-3398. doi: 10.11999/JEIT211517
LIU Yancheng, DONG Zhangwei, ZHU Pengli, LIU Siyuan. Unsupervised Underwater Image Enhancement Based on Feature Disentanglement[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3389-3398. doi: 10.11999/JEIT211517
Citation: LIU Yancheng, DONG Zhangwei, ZHU Pengli, LIU Siyuan. Unsupervised Underwater Image Enhancement Based on Feature Disentanglement[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3389-3398. doi: 10.11999/JEIT211517

基于特征解耦的无监督水下图像增强

doi: 10.11999/JEIT211517
基金项目: 国家自然科学基金(51979021, 51709028),辽宁省自然科学基金(2019JH8, 10100045),中央高校基本科研业务费专项资金(3132019317, 3132022218)
详细信息
    作者简介:

    刘彦呈:男,教授,研究方向为水下环境感知决策控制和船舶电力推进技术

    董张伟:男,硕士生,研究方向为水下图像分析和机器学习

    朱鹏莅:男,博士生,研究方向为水下图像分析和智能控制

    刘厶源:男,副教授,研究方向为图像分析、智能控制和机器学习

    通讯作者:

    朱鹏莅 dlmu.p.l.zhu@gmail.com

  • 中图分类号: TN911.73; TP391.41

Unsupervised Underwater Image Enhancement Based on Feature Disentanglement

Funds: The National Natural Science Foundation of China (51979021, 51709028), The Natural Science Foundation of Liaoning Province (2019JH8, 10100045), The Fundamental Research Funds for the Central Universities (3132019317, 3132022218)
  • 摘要: 水介质的吸收和散射特性致使水下图像存在不同类型的失真,严重影响后续处理的准确性和有效性。目前有监督学习的水下图像增强方法依靠合成的水下配对图像集进行训练,然而由于合成的数据可能无法准确地模拟水下成像的基本物理机制,所以监督学习的方法很难应用于实际的应用场景。该文提出一种基于特征解耦的无监督水下图像增强方法,一方面,考虑获取同一场景下的清晰-非清晰配对数据集难度大且成本高,提出采用循环生成对抗网络将水下图像增强问题转换成风格迁移问题,实现无监督学习;另一方面,结合特征解耦方法分别提取图像的风格特征和结构特征,保证增强前后图像的结构一致性。实验结果表明,该方法可以在非配对数据训练的情况下,能够有效恢复水下图像的颜色和纹理细节。
  • 图  1  基于特征解耦的无监督水下图像网络整体框架

    图  2  生成器网络结构图

    图  3  本文网络损失函数总览图

    图  4  水下图像定性分析对比图

    图  5  消融实验定性分析对比图

    表  1  水下图像定量分析对比

    UICMUISMUIConMUIQMUCIQE
    原图
    IBLA
    3.1494
    5.1639
    5.0460
    4.9656
    0.2046
    0.1268
    2.3105
    2.0654
    4.0049
    5.0761
    SMBLO9.09975.43420.16962.46775.7153
    UGAN
    UWGAN
    FUNIE-GAN
    本文方法
    4.5587
    4.4339
    4.8336
    4.8401
    6.8883
    6.5132
    6.7341
    6.2767
    0.2282
    0.2921
    0.2425
    0.3289
    2.9785
    3.0926
    2.9919
    3.1487
    4.9390
    5.2140
    5.5505
    4.3639
    下载: 导出CSV
  • [1] GHANI A S A and ISA N A M. Enhancement of low quality underwater image through integrated global and local contrast correction[J]. Applied Soft Computing, 2015, 37: 332–344. doi: 10.1016/j.asoc.2015.08.033
    [2] SINGH K, KAPOOR R, and SINHA S K. Enhancement of low exposure images via recursive histogram equalization algorithms[J]. Optik, 2015, 126(20): 2619–2625. doi: 10.1016/j.ijleo.2015.06.060
    [3] ZHANG Shu, WANG Ting, DONG Junyu, et al. Underwater image enhancement via extended multi-scale Retinex[J]. Neurocomputing, 2017, 245: 1–9. doi: 10.1016/j.neucom.2017.03.029
    [4] MERCADO M A, ISHII K, and AHN J. Deep-sea image enhancement using multi-scale retinex with reverse color loss for autonomous underwater vehicles[C]. OCEANS 2017-Anchorage, Anchorage, USA, 2017: 1–6.
    [5] GHANI A S A. Image contrast enhancement using an integration of recursive-overlapped contrast limited adaptive histogram specification and dual-image wavelet fusion for the high visibility of deep underwater image[J]. Ocean Engineering, 2018, 162: 224–238. doi: 10.1016/j.oceaneng.2018.05.027
    [6] LI Chongyi and GUO Jichang. Underwater image enhancement by dehazing and color correction[J]. Journal of Electronic Imaging, 2015, 24(3): 033023. doi: 10.1117/1.JEI.24.3.033023
    [7] JAFFE J S. Computer modeling and the design of optimal underwater imaging systems[J]. IEEE Journal of Oceanic Engineering, 1990, 15(2): 101–111. doi: 10.1109/48.50695
    [8] AKKAYNAK D and TREIBITZ T. A revised underwater image formation model[C]. The IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6723–6732.
    [9] ANWAR S and LI Chongyi. Diving deeper into underwater image enhancement: A survey[J]. Signal Processing:Image Communication, 2020, 89: 115978. doi: 10.1016/j.image.2020.115978
    [10] SUN Xin, LIU Lipeng, LI Qiong, et al. Deep pixel-to-pixel network for underwater image enhancement and restoration[J]. IET Image Processing, 2018, 13(3): 469–474. doi: 10.1049/iet-ipr.2018.5237
    [11] WANG Yang, ZHANG Jing, CAO Yang, et al. A deep CNN method for underwater image enhancement[C]. 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 2017: 1382–1386.
    [12] LI Chongyi, GUO Chunle, REN Wenqi, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2019, 29: 4376–4389. doi: 10.1109/TIP.2019.2955241
    [13] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]. The 27th International Conference on Neural Information Processing Systems, Cambridge, UK, 2014: 2672–2680.
    [14] FABBRI C, ISLAM M J, and SATTAR J. Enhancing underwater imagery using generative adversarial networks[C]. 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 2018: 7159–7165.
    [15] GULRAJANI I, AHMED F, ARJOVSKY M, et al. Improved training of wasserstein GANs[C]. The 31st International Conference on Neural Information Processing Systems, Red Hook, USA, 2017: 5769–5779.
    [16] GUO Yecai, LI Hanyu, and ZHUANG Peixian. Underwater image enhancement using a multiscale dense generative adversarial network[J]. IEEE Journal of Oceanic Engineering, 2020, 45(3): 862–870. doi: 10.1109/JOE.2019.2911447
    [17] LI Jie, SKINNER K A, EUSTICE R M, et al. WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2017, 3(1): 387–394. doi: 10.1109/LRA.2017.2730363
    [18] LU Jingyu, LI Na, ZHANG Shaoyong, et al. Multi-scale adversarial network for underwater image restoration[J]. Optics & Laser Technology, 2019, 110: 105–113. doi: 10.1016/j.optlastec.2018.05.048
    [19] ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. The 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 2242–2251.
    [20] ISLAM M J, XIA Youya, and SATTAR J. Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3227–3234. doi: 10.1109/LRA.2020.2974710
    [21] MIRZA M and OSINDERO S. Conditional generative adversarial nets[J]. arXiv preprint arXiv: 1411.1784, 2014.
    [22] LEE H Y, TSENG H Y, HUANG Jiabin, et al. Diverse image-to-image translation via disentangled representations[C]. The 15th European Conference on Computer Vision (ECCV), Munich, Germany, 2018: 36–52.
    [23] HUANG Xun, LIU Mingyu, BELONGIE S, et al. Multimodal unsupervised image-to-image translation[C]. The 15th European Conference on Computer Vision (ECCV), Munich, Germany, 2018: 179–196.
    [24] LIU Siyuan, THUNG K H, QU Liangqiong, et al. Learning MRI artefact removal with unpaired data[J]. Nature Machine Intelligence, 2021, 3(1): 60–67. doi: 10.1038/s42256-020-00270-2
    [25] LOCATELLO F, BAUER S, LUCIC M, et al. Challenging common assumptions in the unsupervised learning of disentangled representations[C]. The 36th International Conference on Machine Learning, Long Beach, USA, 2019: 4114–4124.
    [26] LI Chuan and WAND M. Precomputed real-time texture synthesis with markovian generative adversarial networks[C]. 14th European Conference on Computer Vision, Amsterdam, Holland, 2016: 702–716.
    [27] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. The 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778.
    [28] LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]. The 2017 IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, 2017: 936–944.
    [29] HUANG Xun and BELONGIE S. Arbitrary style transfer in real-time with adaptive instance normalization[C]. The 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 1510–1519.
    [30] KINGMA D P and WELLING M. Auto-encoding variational bayes[J]. arXiv preprint arXiv: 1312.6114v10, 2013.
    [31] KINGMA D P and BA J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv: 1412.6980v5, 2014.
    [32] PENG Y T and COSMAN P C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579–1594. doi: 10.1109/TIP.2017.2663846
    [33] SONG Wei, WANG Yan, HUANG Dongmei, et al. Enhancement of underwater images with statistical model of background light and optimization of transmission map[J]. IEEE Transactions on Broadcasting, 2020, 66(1): 153–169. doi: 10.1109/TBC.2019.2960942
    [34] LI Chongyi, GUO Jichang, and GUO Chunle. Emerging from water: Underwater image color correction based on weakly supervised color transfer[J]. IEEE Signal Processing Letters, 2018, 25(3): 323–327. doi: 10.1109/LSP.2018.2792050
    [35] YANG Miao and SOWMYA A. An underwater color image quality evaluation metric[J]. IEEE Transactions on Image Processing, 2015, 24(12): 6062–6071. doi: 10.1109/TIP.2015.2491020
    [36] PANETTA K, GAO Chen, and AGAIAN S. Human-visual-system-inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2016, 41(3): 541–551. doi: 10.1109/JOE.2015.2469915
  • 加载中
图(5) / 表(1)
计量
  • 文章访问数:  1522
  • HTML全文浏览量:  701
  • PDF下载量:  250
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-12-15
  • 修回日期:  2022-02-27
  • 录用日期:  2022-03-03
  • 网络出版日期:  2022-03-07
  • 刊出日期:  2022-10-19

目录

    /

    返回文章
    返回