高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于多路混合注意力机制的水下图像增强网络

李云 孙山林 黄晴 井佩光

李云, 孙山林, 黄晴, 井佩光. 基于多路混合注意力机制的水下图像增强网络[J]. 电子与信息学报, 2024, 46(1): 118-128. doi: 10.11999/JEIT230495
引用本文: 李云, 孙山林, 黄晴, 井佩光. 基于多路混合注意力机制的水下图像增强网络[J]. 电子与信息学报, 2024, 46(1): 118-128. doi: 10.11999/JEIT230495
LI Yun, SUN Shanlin, HUANG Qing, JING Peiguang. Underwater Image Enhancement Network Based on Multi-channel Hybrid Attention Mechanism[J]. Journal of Electronics & Information Technology, 2024, 46(1): 118-128. doi: 10.11999/JEIT230495
Citation: LI Yun, SUN Shanlin, HUANG Qing, JING Peiguang. Underwater Image Enhancement Network Based on Multi-channel Hybrid Attention Mechanism[J]. Journal of Electronics & Information Technology, 2024, 46(1): 118-128. doi: 10.11999/JEIT230495

基于多路混合注意力机制的水下图像增强网络

doi: 10.11999/JEIT230495
基金项目: 国家自然科学基金(61861014),博士启动基金(BS2021025)
详细信息
    作者简介:

    李云:女,教授,博士,研究方向为水下无线传感器网络、大数据分析和智能算法

    孙山林:男,教授,博士,研究方向为图像处理和智能算法

    黄晴:男,硕士生,研究方向为图像处理、人工智能

    井佩光:男,副教授,博士,研究方向为多媒体信息处理、图像处理、人工智能

    通讯作者:

    井佩光 pgjing@tju.edu.cn

  • 中图分类号: TN911.73

Underwater Image Enhancement Network Based on Multi-channel Hybrid Attention Mechanism

Funds: The National Natural Science Foundation of China(61861014), Ph. D. Startup Fund(BS2021025)
  • 摘要: 光线在水下被吸收或者散射使得水下图像成像出现色偏、模糊遮挡等问题,影响水下视觉任务。传统的图像增强方法分别采用直方图均衡、伽马矫正和白平衡方法较好地增强水下图像。然而,3种方法融合增强水下图像的互补性和相关性方面的研究较少。因此,该文提出一种基于多路混合注意力机制的水下图像增强网络。首先,提出多路特征提取模块,对图像进行直方图均衡支路、伽马矫正支路和白平衡支路的多路特征提取,提取图像的对比度、亮度和颜色特征;然后,融合直方图均衡、伽马矫正和白平衡3支路特征,增强3支路特征融合的互补性;最后,设计混合注意力学习模块,深度挖掘3支路在对比度、亮度和颜色的相关性矩阵,并引入跳跃连接增强图像输出。在多个数据集上的实验结果表明,该方法能够有效恢复水下图像色偏、模糊遮挡和提高图像明亮度。
  • 图  1  网络结构图

    图  2  消融实验训练损失曲线图

    图  3  UIEB数据集上不同方法生成的水下图像视觉对比

    图  4  EUVP1数据集上不同方法生成的水下图像视觉对比

    图  5  EUVP2数据集上不同方法生成的水下图像视觉对比

    图  6  LSUI数据集上不同方法生成的水下图像视觉对比

    表  1  4种不同水下数据集的划分

    UIEBEUVP1EUVP2LSUI
    总图像890370021854279
    训练集790333019663879
    测试集100370219400
    下载: 导出CSV

    表  2  网络结构消融实验数值结果

    MSE(×103)PSNRSSIM
    A0.932919.55490.8759
    A+B4.654711.84240.6027
    A+C1.088618.61550.8698
    A+D11.60757.64270.1967
    A+B+C0.909619.77090.8864
    A+B+D0.769020.26610.8901
    A+C+D0.806220.36610.8925
    A+B+C+D0.767120.38850.8798
    A+B+C+D+E0.719320.76780.8958
    下载: 导出CSV

    表  3  UIEB数据集上不同方法的定量评价

    MSE(×103)PSNRSSIMUCIQE运行时间(s)
    原图1.516018.37820.77630.3934
    Fusion0.976428.12350.84280.483267.85
    GDCP3.617313.56130.73650.4560579.41
    Dehaze2.398115.79780.75440.462412.74
    HLRP4.669212.18800.23090.4673300.11
    BRUE2.663315.08870.65550.4257249.12
    Shallow-UWnet1.226318.47890.80410.356630.48
    WaterNet1.025818.77040.87770.416564.85
    UResnet1.434317.95620.76150.406534.74
    本文0.719320.76780.89580.407657.85
    下载: 导出CSV

    表  4  EUVP1数据集上不同方法的定量评价

    MSE(×103)PSNRSSIMUCIQE运行时间(s)
    原图1.516216.94080.72820.4244
    Fusion1.554316.54570.66060.471130.87
    GDCP4.353812.43840.59220.4735223.98
    Dehaze2.995014.06050.64220.45757.87
    HLRP4.195312.26900.12950.4607144.62
    BRUE1.973716.39210.65040.428185.99
    Shallow-UWnet0.394522.77180.80850.39385.28
    WaterNet0.413322.87060.83390.41286.90
    UResnet0.323723.78040.82960.43708.81
    本文0.365123.38990.83090.43787.49
    下载: 导出CSV

    表  5  EUVP2数据集上不同方法的定量评价

    MSE(×103)PSNRSSIMUCIQE运行时间(s)
    原图0.764419.84640.73610.4274
    Fusion1.528716.85240.66820.482219.44
    GDCP3.211213.70860.62270.4794149.14
    Dehaze2.075216.26890.67050.46465.25
    HLRP4.060412.30130.16690.471895.323
    BRUE2.033915.95200.66260.432458.81
    Shallow-UWnet0.323423.56120.77800.41284.71
    WaterNet0.464922.34870.79970.420710.00
    UResnet0.298224.08100.80340.431410.11
    本文0.373223.14600.80550.40635.40
    下载: 导出CSV

    表  6  LSUI数据集上不同方法的定量评价

    MSE(×103)PSNRSSIMUCIQE运行时间(s)
    原图1.068418.80530.79770.4194
    Fusion1.324217.66040.72770.482534.72
    GDCP3.545913.40690.67660.4796363.02
    Dehaze2.382815.46750.71920.46749.64
    HLRP3.894912.51320.19180.4915240.86
    BRUE1.755816.67390.70740.4301162.92
    Shallow-UWnet0.852719.66730.80460.37099.24
    WaterNet0.493122.13360.86280.437013.18
    UResnet0.583121.15760.81240.424313.13
    本文0.557621.55630.85520.400415.50
    下载: 导出CSV

    表  7  不同方法的运行时间和FPS

    FusionGDCPDehazeHLRPBRUEShallow-UWnetWaterNetUResnet本文
    时间(s)67.85579.4112.74300.11249.1230.4864.8534.7457.85
    Fps1.470.177.850.330.403.281.542.881.73
    下载: 导出CSV
  • [1] 侯冬, 任军委, 郭广坤, 等. 高精度水下激光频率传递研究进展[J]. 光电工程, 2023, 50(2): 220149. doi: 10.12086/oee.2023.220149

    HOU Dong, REN Junwei, GUO Guangkun, et al. Progress on high-precision laser-based underwater frequency transfer[J]. Opto-Electronic Engineering, 2023, 50(2): 220149. doi: 10.12086/oee.2023.220149
    [2] PIZER S M, AMBURN E P, AUSTIN J D, et al. Adaptive histogram equalization and its variations[J]. Computer Vision, Graphics, and Image Processing, 1987, 39(3): 355–368. doi: 10.1016/S0734-189X(87)80186-X
    [3] PIZER S M, JOHNSTON R E, ERICKSEN J P, et al. Contrast-limited adaptive histogram equalization: Speed and effectiveness[C]. The First Conference on Visualization in Biomedical Computing, Atlanta, USA, 1990: 337–345.
    [4] ANCUTI C, ANCUTI C O, HABER T, et al. Enhancing underwater images and videos by fusion[C]. 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, USA, 2012: 81–88.
    [5] RIZZI A, GATTA C, and MARINI D. Color correction between gray world and white patch[C]. The SPIE 4662, Human Vision and Electronic Imaging VII, San Jose, USA, 2002: 1–10.
    [6] SINGH G, JAGGI N, VASAMSETTI S, et al. Underwater image/video enhancement using wavelet based color correction (WBCC) method[C]. 2015 IEEE Underwater Technology, Chennai, India, 2015: 1–5.
    [7] MUNIRAJ M and DHANDAPANI V. Underwater image enhancement by combining color constancy and dehazing based on depth estimation[J]. Neurocomputing, 2021, 460: 211–230. doi: 10.1016/j.neucom.2021.07.003
    [8] BERMAN D, LEVY D, AVIDAN S, et al. Underwater single image color restoration using haze-lines and a new quantitative dataset[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(8): 2822–2837. doi: 10.1109/TPAMI.2020.2977624
    [9] WANG Yi, LIU Hui, and CHAU L P. Single underwater image restoration using adaptive attenuation-curve prior[J]. IEEE Transactions on Circuits and Systems I:Regular Papers, 2018, 65(3): 992–1002. doi: 10.1109/TCSI.2017.2751671
    [10] PENG Y T, CAO Keming, and COSMAN P C. Generalization of the dark channel prior for single image restoration[J]. IEEE Transactions on Image Processing, 2018, 27(6): 2856–2868. doi: 10.1109/TIP.2018.2813092
    [11] LI Chongyi, ANWAR S, HOU Junhui, et al. Underwater image enhancement via medium transmission-guided multi-color space embedding[J]. IEEE Transactions on Image Processing, 2021, 30: 4985–5000. doi: 10.1109/TIP.2021.3076367
    [12] 李钰, 杨道勇, 刘玲亚, 等. 利用生成对抗网络实现水下图像增强[J]. 上海交通大学学报, 2022, 56(2): 134–142. doi: 10.16183/j.cnki.jsjtu.2021.075

    LI Yu, YANG Daoyong, LIU Lingya, et al. Underwater image enhancement based on generative adversarial networks[J]. Journal of Shanghai Jiaotong University, 2022, 56(2): 134–142. doi: 10.16183/j.cnki.jsjtu.2021.075
    [13] MARQUES T P and ALBU A B. L2UWE: A framework for the efficient enhancement of low-light underwater images using local contrast and multi-scale fusion[C]. The 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, USA, 2020: 2286–2295.
    [14] LI Chongyi, GUO Chunle, REN Wenqi, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020, 29: 4376–4389. doi: 10.1109/TIP.2019.2955241
    [15] SUN Xin, LIU Lipeng, LI Qiong, et al. Deep pixel-to-pixel network for underwater image enhancement and restoration[J]. IET Image Processing, 2018, 13(3): 469–474. doi: 10.1049/iet-ipr.2018.5237
    [16] 方明, 刘小晗, 付飞蚺. 基于注意力的多尺度水下图像增强网络[J]. 电子与信息学报, 2021, 43(12): 3513–3521. doi: 10.11999/JEIT200836

    FANG Ming, LIU Xiaohan, and FU Feiran. Multi-scale underwater image enhancement network based on attention mechanism[J]. Journal of Electronics &Information Technology, 2021, 43(12): 3513–3521. doi: 10.11999/JEIT200836
    [17] 米泽田, 晋洁, 李圆圆, 等. 基于多尺度级联网络的水下图像增强方法[J]. 电子与信息学报, 2022, 44(10): 3353–3362. doi: 10.11999/JEIT220375

    MI Zetian, JIN Jie, LI Yuanyuan, et al. Underwater image enhancement method based on multi-scale cascade network[J]. Journal of Electronics &Information Technology, 2022, 44(10): 3353–3362. doi: 10.11999/JEIT220375
    [18] ZHUANG Peixian, WU Jiamin, PORIKLI F, et al. Underwater image enhancement with hyper-laplacian reflectance priors[J]. IEEE Transactions on Image Processing, 2022, 31: 5442–5455. doi: 10.1109/TIP.2022.3196546
    [19] ZHUANG Peixian, LI Chongyi, and WU Jiamin. Bayesian retinex underwater image enhancement[J]. Engineering Applications of Artificial Intelligence, 2021, 101: 104171. doi: 10.1016/j.engappai.2021.104171
    [20] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 2818–2826.
    [21] BAHDANAU D, CHO K, and BENGIO Y. Neural machine translation by jointly learning to align and translate[C]. The 3rd International Conference on Learning Representations, San Diego, USA, 2014.
    [22] WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.1109/TIP.2003.819861
    [23] ISLAM M J, XIA Youya, and SATTAR J. Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3227–3234. doi: 10.1109/LRA.2020.2974710
    [24] PENG Lintao, ZHU Chunli, and BIAN Liheng. U-shape transformer for underwater image enhancement[J]. IEEE Transactions on Image Processing, 2023, 32: 3066–3079. doi: 10.1109/TIP.2023.3276332
    [25] YANG Miao and SOWMYA A. An underwater color image quality evaluation metric[J]. IEEE Transactions on Image Processing, 2015, 24(12): 6062–6071. doi: 10.1109/TIP.2015.2491020
    [26] HE Kaiming, SUN Jian, and TANG Xiaoou. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341–2353. doi: 10.1109/TPAMI.2010.168
    [27] LIU Peng, WANG Guoyu, QI Hao, et al. Underwater image enhancement with a deep residual framework[J]. IEEE Access, 2019, 7: 94614–94629. doi: 10.1109/ACCESS.2019.2928976
    [28] NAIK A, SWARNAKAR A, and MITTAL K. Shallow-UWnet: Compressed model for underwater image enhancement (student abstract)[C]. The 35th AAAI Conference on Artificial Intelligence, 2021: 15853–15854.
  • 加载中
图(6) / 表(7)
计量
  • 文章访问数:  210
  • HTML全文浏览量:  127
  • PDF下载量:  65
  • 被引次数: 0
出版历程
  • 收稿日期:  2023-05-26
  • 修回日期:  2023-06-28
  • 网络出版日期:  2023-07-03
  • 刊出日期:  2024-01-17

目录

    /

    返回文章
    返回