高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于密度分类引导的双阶段雨天图像复原方法

梅天灿 曹敏 杨宏 高智 易国洪

梅天灿, 曹敏, 杨宏, 高智, 易国洪. 基于密度分类引导的双阶段雨天图像复原方法[J]. 电子与信息学报, 2023, 45(4): 1383-1390. doi: 10.11999/JEIT220157
引用本文: 梅天灿, 曹敏, 杨宏, 高智, 易国洪. 基于密度分类引导的双阶段雨天图像复原方法[J]. 电子与信息学报, 2023, 45(4): 1383-1390. doi: 10.11999/JEIT220157
MEI Tiancan, CAO Min, YANG Hong, GAO Zhi, YI Guohong. Two-stage Rain Image Removal Based on Density Guidance[J]. Journal of Electronics & Information Technology, 2023, 45(4): 1383-1390. doi: 10.11999/JEIT220157
Citation: MEI Tiancan, CAO Min, YANG Hong, GAO Zhi, YI Guohong. Two-stage Rain Image Removal Based on Density Guidance[J]. Journal of Electronics & Information Technology, 2023, 45(4): 1383-1390. doi: 10.11999/JEIT220157

基于密度分类引导的双阶段雨天图像复原方法

doi: 10.11999/JEIT220157
详细信息
    作者简介:

    梅天灿:男,副教授,研究方向为计算机视觉、模式识别、机器学习

    曹敏:女,硕士生,研究方向为恶劣天气图像复原

    杨宏:男,高级工程师,研究方向为航空遥感及数据应用技术

    高智:男,教授,研究方向为人工智能、计算机视觉、智能无人系统、遥感

    易国洪:男,副教授,研究方向为软件工程、模式识别

    通讯作者:

    梅天灿 mtc@whu.edu.cn

  • 中图分类号: TN911.73

Two-stage Rain Image Removal Based on Density Guidance

  • 摘要: 雨天作为最常见的恶劣天气,对图像造成的退化效应主要包括雨线对背景的遮挡、雨线累积形成的雨雾效应,从而导致很多为清晰成像条件设计的视觉系统运行效果大打折扣。为了实现雨线和雨雾同时去除、更鲁棒地处理各种真实雨天场景,该文提出了一种雨密度分类引导的双阶段雨天图像复原方法。该方法结合雨天物理模型先验与cGAN网络优化,综合考虑不同模式的雨线与雨雾,利用单独的雨密度分类网络为优化阶段提供引导信息,可以实现不同密度的雨线和雨雾图像复原。在公开合成数据集和真实雨天图像上进行了大量实验,定量和定性的结果均表明了所提方法在去雨有效性和泛化性上的优势。
  • 图  1  基于密度分类引导的双阶段去雨网络结构

    图  2  先验去雨网络子网络结构

    图  3  不同密度大小雨天图像及其真值

    图  4  不同方法在合成测试图像上去雨结果

    图  5  不同方法在真实雨天图像上去雨结果

    图  6  本文算法去雨前后图像目标检测结果

    表  1  不同方法在OTS-test测试集上的平均PSNR和SSIM结果

    方法PSNR (dB)SSIM
    DID-MDN18.970.7386
    HRR23.670.8720
    PReNet23.860.8782
    MSPFN23.690.8607
    本文方法25.100.8854
    下载: 导出CSV

    表  2  不同方法在Rain100L和Rain100L测试集上的平均PSNR和SSIM结果

    方法Rain100LRain100H
    PSNR (dB)SSIMPSNR (dB)SSIM
    DSC24.160.8715.660.54
    LP29.110.8814.260.42
    DID-MDN25.560.8615.950.56
    HRR25.400.8922.190.78
    本文方法29.210.9224.140.83
    下载: 导出CSV

    表  3  不同网络结构OTS-test测试集上的平均PSNR和SSIM结果

    方法PSNR (dB)SSIM
    DID-MDN18.970.738 6
    DID-MDN+优化19.580.754 6
    本文方法-分类器23.920.866 6
    本文方法25.100.885 4
    下载: 导出CSV
  • [1] BAHNSEN C H and MOESLUND T B. Rain removal in traffic surveillance: Does it matter?[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 20(8): 2802–2819. doi: 10.1109/TITS.2018.2872502
    [2] GARG K and NAYAR S K. Vision and rain[J]. International Journal of Computer Vision, 2007, 75(1): 3–27. doi: 10.1007/s11263-006-0028-6
    [3] LI Yu, TAN R T, GUO Xiaojie, et al. Rain streak removal using layer priors[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 2736–2744.
    [4] CHEN Yilei and HSU C T. A generalized low-rank appearance model for spatio-temporally correlated rain streaks[C]. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia, 2013: 1968–1975.
    [5] 王志超, 陈震. 基于小波融合的视频图像去雨(雪)方法[J]. 北华大学学报:自然科学版, 2018, 19(1): 135–140. doi: 10.11713/j.issn.1009-4822.2018.01.028

    WANG Zhichao and CHEN Zhen. Method of removing rain (snow) from video images based on wavelet fusion[J]. Journal of Beihua University:Natural Science, 2018, 19(1): 135–140. doi: 10.11713/j.issn.1009-4822.2018.01.028
    [6] FU Xueyang, HUANG Jiabin, DING Xinghao, et al. Clearing the skies: A deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing, 2017, 26(6): 2944–2956. doi: 10.1109/TIP.2017.2691802
    [7] 郭继昌, 郭昊, 郭春乐. 多尺度卷积神经网络的单幅图像去雨方法[J]. 哈尔滨工业大学学报, 2018, 50(3): 185–191. doi: 10.11918/j.issn.0367-6234.201704075

    GUO Jichang, GUO Hao, and GUO Chunle. Single image rain removal based on multi-scale convolutional neural network[J]. Journal of Harbin Institute of Technology, 2018, 50(3): 185–191. doi: 10.11918/j.issn.0367-6234.201704075
    [8] ZHANG He and PATEL V M. Density-aware single image de-raining using a multi-stream dense network[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 695–704.
    [9] REN Dongwei, ZUO Wangmeng, HU Qinghua, et al. Progressive image deraining networks: A better and simpler baseline[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 3932–3941.
    [10] JIANG Kui, WANG Zhongyuan, YI Peng, et al. Multi-scale progressive fusion network for single image deraining[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 8343–8352.
    [11] LIN C Y, TAO Zhuang, XU Aisheng, et al. Sequential dual attention network for rain streak removal in a single image[J]. IEEE Transactions on Image Processing, 2020, 29: 9250–9265. doi: 10.1109/TIP.2020.3025402
    [12] DENG Sen, WEI Mingqiang, WANG Jun, et al. Detail-recovery image deraining via context aggregation networks[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 14548–14557.
    [13] FU Xueyang, QI Qi, ZHA Zhengjun, et al. Rain streak removal via dual graph convolutional network[C]. Proceedings of the AAAI Conference on Artificial Intelligence, Palo Alto, USA, 2021: 1352–1360.
    [14] GUO Qing, SUN Jingyang, JUEFEI-XU F, et al. Uncertainty-aware cascaded dilation filtering for high-efficiency deraining[J]. arXiv: 2201.02366, 2022.
    [15] ZHENG Shen, LU Changjie, WU Yuxiong, et al. SAPNet: Segmentation-aware progressive network for perceptual contrastive deraining[C]. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa, USA, 2022: 52–62.
    [16] LI Ruoteng, CHEONG L F, and TAN R T. Heavy rain image restoration: Integrating physics model and conditional adversarial learning[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 1633–1642.
    [17] WEI Yanyan, ZHANG Zhao, WANG Yang, et al. Deraincyclegan: Rain attentive cyclegan for single image deraining and rainmaking[J]. IEEE Transactions on Image Processing, 2021, 30: 4788–4801. doi: 10.1109/TIP.2021.3074804
    [18] WEI Yanyan, ZHANG Zhao, WANG Yang, et al. Semi-deraingan: A new semi-supervised single image deraining[C]. IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, China, 2021: 1–6.
    [19] ISOLA P, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 5967–5976.
    [20] JOLICOEUR-MARTINEAU A. The relativistic discriminator: A key element missing from standard GAN[C]. 7th International Conference on Learning Representations, New Orleans, USA, 2018.
    [21] LI Boyi, REN Wenqi, FU Dengpan, et al. Benchmarking single-image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2019, 28(1): 492–505. doi: 10.1109/TIP.2018.2867951
    [22] BOCHKOVSKIY A, WANG C Y, and LIAO H Y M. Yolov4: Optimal speed and accuracy of object detection[J]. arXiv: 2004.10934, 2020.
  • 加载中
图(6) / 表(3)
计量
  • 文章访问数:  405
  • HTML全文浏览量:  366
  • PDF下载量:  91
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-02-21
  • 修回日期:  2022-08-31
  • 网络出版日期:  2022-09-03
  • 刊出日期:  2023-04-10

目录

    /

    返回文章
    返回