Two-stage Rain Image Removal Based on Density Guidance

MEI Tiancan; CAO Min; YANG Hong; GAO Zhi; YI Guohong

doi:10.11999/JEIT220157

Volume 45 Issue 4

Apr. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 > 45(4): 1383-1390

MEI Tiancan, CAO Min, YANG Hong, GAO Zhi, YI Guohong. Two-stage Rain Image Removal Based on Density Guidance[J]. Journal of Electronics & Information Technology, 2023, 45(4): 1383-1390. doi: 10.11999/JEIT220157

Citation:

MEI Tiancan, CAO Min, YANG Hong, GAO Zhi, YI Guohong. Two-stage Rain Image Removal Based on Density Guidance[J]. Journal of Electronics & Information Technology, 2023, 45(4): 1383-1390. doi: 10.11999/JEIT220157

Citation:

PDF( 8329 KB)

Two-stage Rain Image Removal Based on Density Guidance

doi: 10.11999/JEIT220157

1.
School of Electronic Information, Wuhan University, Wuhan 430072, China
2.
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China
3.
School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430072, China
4.
School of Computer Science and Engineering, Wuhan Institute of Technology, Wuhan 430205, China

Received Date: 2022-02-21
Rev Recd Date: 2022-08-31

Available Online: 2022-09-03

Publish Date: 2023-04-10

Abstract

Abstract

As the most common severe weather, rain can degrade the performance of many vision systems designed for clear imaging conditions. In order to realize the simultaneous removal of rain streaks and rain accumulation, and to deal with various real rain scenes, a two-stage rain image restoration method guided by rain density classification is proposed, which integrates physics model and cGAN refinement. Extensive experiments are conducted on representative synthetic rain datasets and realrain scenes. Quantitative and qualitative results demonstrate the superiority of the proposed method in terms of effectiveness and generalization ability.
- Image deraining,
- Physics model,
- cGAN,
- Density-aware,
- Gated fusion

FullText(HTML)

References(22)

References

[1]	BAHNSEN C H and MOESLUND T B. Rain removal in traffic surveillance: Does it matter?[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 20(8): 2802–2819. doi: 10.1109/TITS.2018.2872502
[2]	GARG K and NAYAR S K. Vision and rain[J]. International Journal of Computer Vision, 2007, 75(1): 3–27. doi: 10.1007/s11263-006-0028-6
[3]	LI Yu, TAN R T, GUO Xiaojie, et al. Rain streak removal using layer priors[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 2736–2744.
[4]	CHEN Yilei and HSU C T. A generalized low-rank appearance model for spatio-temporally correlated rain streaks[C]. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia, 2013: 1968–1975.
[5]	王志超, 陈震. 基于小波融合的视频图像去雨(雪)方法[J]. 北华大学学报:自然科学版, 2018, 19(1): 135–140. doi: 10.11713/j.issn.1009-4822.2018.01.028 WANG Zhichao and CHEN Zhen. Method of removing rain (snow) from video images based on wavelet fusion[J]. Journal of Beihua University:Natural Science, 2018, 19(1): 135–140. doi: 10.11713/j.issn.1009-4822.2018.01.028
[6]	FU Xueyang, HUANG Jiabin, DING Xinghao, et al. Clearing the skies: A deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing, 2017, 26(6): 2944–2956. doi: 10.1109/TIP.2017.2691802
[7]	郭继昌, 郭昊, 郭春乐. 多尺度卷积神经网络的单幅图像去雨方法[J]. 哈尔滨工业大学学报, 2018, 50(3): 185–191. doi: 10.11918/j.issn.0367-6234.201704075 GUO Jichang, GUO Hao, and GUO Chunle. Single image rain removal based on multi-scale convolutional neural network[J]. Journal of Harbin Institute of Technology, 2018, 50(3): 185–191. doi: 10.11918/j.issn.0367-6234.201704075
[8]	ZHANG He and PATEL V M. Density-aware single image de-raining using a multi-stream dense network[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 695–704.
[9]	REN Dongwei, ZUO Wangmeng, HU Qinghua, et al. Progressive image deraining networks: A better and simpler baseline[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 3932–3941.
[10]	JIANG Kui, WANG Zhongyuan, YI Peng, et al. Multi-scale progressive fusion network for single image deraining[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 8343–8352.
[11]	LIN C Y, TAO Zhuang, XU Aisheng, et al. Sequential dual attention network for rain streak removal in a single image[J]. IEEE Transactions on Image Processing, 2020, 29: 9250–9265. doi: 10.1109/TIP.2020.3025402
[12]	DENG Sen, WEI Mingqiang, WANG Jun, et al. Detail-recovery image deraining via context aggregation networks[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 14548–14557.
[13]	FU Xueyang, QI Qi, ZHA Zhengjun, et al. Rain streak removal via dual graph convolutional network[C]. Proceedings of the AAAI Conference on Artificial Intelligence, Palo Alto, USA, 2021: 1352–1360.
[14]	GUO Qing, SUN Jingyang, JUEFEI-XU F, et al. Uncertainty-aware cascaded dilation filtering for high-efficiency deraining[J]. arXiv: 2201.02366, 2022.
[15]	ZHENG Shen, LU Changjie, WU Yuxiong, et al. SAPNet: Segmentation-aware progressive network for perceptual contrastive deraining[C]. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa, USA, 2022: 52–62.
[16]	LI Ruoteng, CHEONG L F, and TAN R T. Heavy rain image restoration: Integrating physics model and conditional adversarial learning[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 1633–1642.
[17]	WEI Yanyan, ZHANG Zhao, WANG Yang, et al. Deraincyclegan: Rain attentive cyclegan for single image deraining and rainmaking[J]. IEEE Transactions on Image Processing, 2021, 30: 4788–4801. doi: 10.1109/TIP.2021.3074804
[18]	WEI Yanyan, ZHANG Zhao, WANG Yang, et al. Semi-deraingan: A new semi-supervised single image deraining[C]. IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, China, 2021: 1–6.
[19]	ISOLA P, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 5967–5976.
[20]	JOLICOEUR-MARTINEAU A. The relativistic discriminator: A key element missing from standard GAN[C]. 7th International Conference on Learning Representations, New Orleans, USA, 2018.
[21]	LI Boyi, REN Wenqi, FU Dengpan, et al. Benchmarking single-image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2019, 28(1): 492–505. doi: 10.1109/TIP.2018.2867951
[22]	BOCHKOVSKIY A, WANG C Y, and LIAO H Y M. Yolov4: Optimal speed and accuracy of object detection[J]. arXiv: 2004.10934, 2020.