Research on Rain Removal Method for Single Image Based on Multi-channel and Multi-scale CNN

Changyuan LIU; Qi WANG; Xiaojun BI

doi:10.11999/JEIT190755

Volume 42 Issue 9

Sep. 2020

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2020 > 42(9): 2285-2292

Changyuan LIU, Qi WANG, Xiaojun BI. Research on Rain Removal Method for Single Image Based on Multi-channel and Multi-scale CNN[J]. Journal of Electronics & Information Technology, 2020, 42(9): 2285-2292. doi: 10.11999/JEIT190755

Citation:

Changyuan LIU, Qi WANG, Xiaojun BI. Research on Rain Removal Method for Single Image Based on Multi-channel and Multi-scale CNN[J]. Journal of Electronics & Information Technology, 2020, 42(9): 2285-2292. doi: 10.11999/JEIT190755

Citation:

PDF( 8551 KB)

Research on Rain Removal Method for Single Image Based on Multi-channel and Multi-scale CNN

doi: 10.11999/JEIT190755

1.
School of Electrical and Electronic Engineering, Harbin University of Science and Technology, Harbin 150000, China
2.
School of Information and Communication Engineering, Harbin Engineering University, Harbin 150000, China

Funds: The National Natural Science Foundation of China(51779050)

Received Date: 2019-09-29
Rev Recd Date: 2020-05-28

Available Online: 2020-07-13

Publish Date: 2020-09-27

Abstract

Abstract

Rainy days and other severe weather will seriously affect the image quality, thus affecting the performance of vision processing algorithms. In order to improve the imaging quality of rain images, a rain removal algorithm based on multi-channel multi-scale convolution neural network to extract rain line features is proposed. Firstly, the rain images are decomposed by wavelet threshold-guided bilateral filtering to obtain high-frequency rain line images and low-frequency background images with high contour preservation. Then, in order to make the rain line information in the high-frequency part of the image more obvious and reduce the background misjudgment in the high-frequency image during the rain line feature learning, the obtained high-frequency rain line image is passed through a filter again to obtain a higher-frequency rain line image with reduced background information and enhanced rain line information. Secondly, in view of the large amount of raindrop imprint left on the low-frequency background image, it is proposed to send the low-frequency background image and the higher-frequency rain line image together into the convolution neural network for feature learning, in which multi-scale feature information is extracted from the image, finally, a more complete restoration image with rain line removal is obtained. At the same time, when constructing the network model, hole convolution is used instead of standard convolution to extract the feature information of the image, thus obtaining richer image features and improving the rain removal performance of the algorithm. From the experimental results, after removing rain, the image is clear and the detail retention is high.
- Deep learning,
- Dilated convolution,
- Image decomposition,
- Multi-scale feature extraction

FullText(HTML)

References(18)

References

郭智, 宋萍, 张义, 等. 基于深度卷积神经网络的遥感图像飞机目标检测方法[J]. 电子与信息学报, 2018, 40(11): 2684–2690. doi: 10.11999/JEIT180117

GUO Zhi, SONG Ping, ZHANG Yi, et al. Aircraft detection method based on deep convolutional neural network for remote sensing images[J]. Journal of Electronics &Information Technology, 2018, 40(11): 2684–2690. doi: 10.11999/JEIT180117

孙彦景, 石韫开, 云霄, 等. 基于多层卷积特征的自适应决策融合目标跟踪算法[J]. 电子与信息学报, 2019, 41(10): 2464–2470. doi: 10.11999/JEIT180971

SUN Yanjing, SHI Yunkai, YUN Xiao, et al. Adaptive strategy fusion target tracking based on multi-layer convolutional features[J]. Journal of Electronics &Information Technology, 2019, 41(10): 2464–2470. doi: 10.11999/JEIT180971

GARG K and NAYAR S K. Detection and removal of rain from videos[C]. 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington DC, USA, 2004. doi: 10.1109/CVPR.2004.1315077.

BARNUM P C, NARASIMHAN S, and KANADE T. Analysis of rain and snow in frequency space[J]. International Journal of Computer Vision, 2010, 86(2): 256–274. doi: 10.1007/s11263-008-0200-2

LIU Jiaying, YANG Wenhan, YANG Shuai, et al. Erase or fill? Deep joint recurrent rain removal and reconstruction in videos[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 3233–3242. doi: 10.1109/CVPR.2018.00341.

CHEN Jie, TAN C H, HOU Junhui, et al. Robust video content alignment and compensation for rain removal in a CNN framework[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6286–6295. doi: 10.1109/CVPR.2018.00658.

KIM J H, LEE C, SIM J Y, et al. Single-image deraining using an adaptive nonlocal means filter[C]. 2013 IEEE International Conference on Image Processing, Melbourne, Australia, 2013: 914–917. doi: 10.1109/ICIP.2013.6738189.

HUANG Dean, KANG Liwei, WANG Y C F, et al. Self-learning based image decomposition with applications to single image denoising[J]. IEEE Transactions on Multimedia, 2014, 16(1): 83–93. doi: 10.1109/TMM.2013.2284759

LUO Yu, XU Yong, and JI Hui. Removing rain from a single image via discriminative sparse coding[C]. 2015 IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 3397–3405. doi: 10.1109/ICCV.2015.388.

KANG Liwei, LIN C W, and FU Y H. Automatic single-image-based rain streaks removal via image decomposition[J]. IEEE Transactions on Image Processing, 2012, 21(4): 1742–1755. doi: 10.1109/TIP.2011.2179057

QIAN Rui, TAN R T, YANG Wenhan, et al. Attentive generative adversarial network for raindrop removal from a single image[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 2482–2491. doi: 10.1109/CVPR.2018.00263.

HE Zhang and PATEL V M. Density-aware single image de-raining using a multi-stream dense network[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 695–704. doi: 10.1109/CVPR.2018.00079.

FU Xueyang, HUANG Jiabin, DING Xinghao, et al. Clearing the skies: A deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing, 2017, 26(6): 2944–2956. doi: 10.1109/TIP.2017.2691802

肖进胜, 李文昊, 姜红, 等. 基于双域滤波的三维块匹配视频去噪算法[J]. 通信学报, 2015, 36(9): 91–97. doi: 10.11959/j.issn.1000-436x.2015245

XIAO Jinsheng, LI Wenhao, JIANG Hong, et al. Three dimensional block-matching video denoising algorithm based on dual-domain filtering[J]. Journal on Communications, 2015, 36(9): 91–97. doi: 10.11959/j.issn.1000-436x.2015245

XIAO Jinsheng, LI Wenhao, LIU Guoxiong, et al. Hierarchical tone mapping based on image colour appearance model[J]. IET Computer Vision, 2014, 8(4): 358–364. doi: 10.1049/iet-cvi.2013.0230

YU Hancheng, LI Zhao, and WANG Haixian. Image denoising using trivariate shrinkage filter in the wavelet domain and joint bilateral filter in the Spatial Domain[J]. IEEE Transactions on Image Processing, 2009, 18(10): 2364–2369. doi: 10.1109/TIP.2009.2026685

HUYNH-THU Q and GHANBARI M. Scope of validity of PSNR in image/video quality assessment[J]. Electronics Letters, 2008, 44(13): 800–801. doi: 10.1049/el:20080522

WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.1109/TIP.2003.819861

Relative Articles

Supplements(0)

Cited By

Proportional views