Research on Rain Removal Method for Single Image Based on Multi-channel and Multi-scale CNN
-
摘要: 雨天等恶劣天气会严重影响到图像成像质量，从而影响到视觉处理算法的性能。为了改善雨天图像的成像质量，该文提出一种基于多通道多尺度卷积神经网络的去雨算法，建立了多通道多尺度卷积神经网络结构来提取雨线特征。首先利用小波阈值引导的双边滤波将有雨图像进行分解，得到高频雨线图像和轮廓保持度高的低频背景图像。然后为了使图像高频部分的雨线信息更为明显，减少雨线特征学习时高频图像中的背景误判，将得到的高频雨线图像再一次通过滤波器得到减弱背景信息同时增强雨线信息的到更高频雨线图像。其次针对低频背景图像上也残留了大量雨痕，该文提出将低频背景图像和更高频雨线图像一起送入卷积神经网络进行特征学习，其中对图像提取的是多尺度特征信息，最后得到雨线去除更彻底的复原图像。同时在构造网络模型时利用空洞卷积代替标准卷积来提取图像的特征信息，得到更丰富的图像特征，提高了算法的去雨性能。从实验结果可以看出去雨之后的图像清晰，细节保持度较高。Abstract: Rainy days and other severe weather will seriously affect the image quality, thus affecting the performance of vision processing algorithms. In order to improve the imaging quality of rain images, a rain removal algorithm based on multi-channel multi-scale convolution neural network to extract rain line features is proposed. Firstly, the rain images are decomposed by wavelet threshold-guided bilateral filtering to obtain high-frequency rain line images and low-frequency background images with high contour preservation. Then, in order to make the rain line information in the high-frequency part of the image more obvious and reduce the background misjudgment in the high-frequency image during the rain line feature learning, the obtained high-frequency rain line image is passed through a filter again to obtain a higher-frequency rain line image with reduced background information and enhanced rain line information. Secondly, in view of the large amount of raindrop imprint left on the low-frequency background image, it is proposed to send the low-frequency background image and the higher-frequency rain line image together into the convolution neural network for feature learning, in which multi-scale feature information is extracted from the image, finally, a more complete restoration image with rain line removal is obtained. At the same time, when constructing the network model, hole convolution is used instead of standard convolution to extract the feature information of the image, thus obtaining richer image features and improving the rain removal performance of the algorithm. From the experimental results, after removing rain, the image is clear and the detail retention is high.
-
表 3 图像质量指标对比
图像 指标 单尺度卷积
5 × 5单尺度卷积
7 × 7多尺度卷积 第1幅 PSNR 21.253 20.197 23.128 SSIM 0.9109 0.9114 0.9236 第2幅 PSNR 25.017 25.714 26.821 SSIM 0.9211 0.9237 0.9420 第3幅 PSNR 31.581 30.336 34.460 SSIM 0.9382 0.9369 0.9448 300张平均 PSNR 25.973 25.285 27.794 SSIM 0.9350 0.9337 0.9434 
下载: 导出CSV
-
郭智, 宋萍, 张义, 等. 基于深度卷积神经网络的遥感图像飞机目标检测方法[J]. 电子与信息学报, 2018, 40(11): 2684–2690. doi: 10.11999/JEIT180117GUO Zhi, SONG Ping, ZHANG Yi, et al. Aircraft detection method based on deep convolutional neural network for remote sensing images[J]. Journal of Electronics &Information Technology, 2018, 40(11): 2684–2690. doi: 10.11999/JEIT180117 孙彦景, 石韫开, 云霄, 等. 基于多层卷积特征的自适应决策融合目标跟踪算法[J]. 电子与信息学报, 2019, 41(10): 2464–2470. doi: 10.11999/JEIT180971SUN Yanjing, SHI Yunkai, YUN Xiao, et al. Adaptive strategy fusion target tracking based on multi-layer convolutional features[J]. Journal of Electronics &Information Technology, 2019, 41(10): 2464–2470. doi: 10.11999/JEIT180971 GARG K and NAYAR S K. Detection and removal of rain from videos[C]. 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington DC, USA, 2004. doi: 10.1109/CVPR.2004.1315077. BARNUM P C, NARASIMHAN S, and KANADE T. Analysis of rain and snow in frequency space[J]. International Journal of Computer Vision, 2010, 86(2): 256–274. doi: 10.1007/s11263-008-0200-2 LIU Jiaying, YANG Wenhan, YANG Shuai, et al. Erase or fill? Deep joint recurrent rain removal and reconstruction in videos[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 3233–3242. doi: 10.1109/CVPR.2018.00341. CHEN Jie, TAN C H, HOU Junhui, et al. Robust video content alignment and compensation for rain removal in a CNN framework[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6286–6295. doi: 10.1109/CVPR.2018.00658. KIM J H, LEE C, SIM J Y, et al. Single-image deraining using an adaptive nonlocal means filter[C]. 2013 IEEE International Conference on Image Processing, Melbourne, Australia, 2013: 914–917. doi: 10.1109/ICIP.2013.6738189. HUANG Dean, KANG Liwei, WANG Y C F, et al. Self-learning based image decomposition with applications to single image denoising[J]. IEEE Transactions on Multimedia, 2014, 16(1): 83–93. doi: 10.1109/TMM.2013.2284759 LUO Yu, XU Yong, and JI Hui. Removing rain from a single image via discriminative sparse coding[C]. 2015 IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 3397–3405. doi: 10.1109/ICCV.2015.388. KANG Liwei, LIN C W, and FU Y H. Automatic single-image-based rain streaks removal via image decomposition[J]. IEEE Transactions on Image Processing, 2012, 21(4): 1742–1755. doi: 10.1109/TIP.2011.2179057 QIAN Rui, TAN R T, YANG Wenhan, et al. Attentive generative adversarial network for raindrop removal from a single image[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 2482–2491. doi: 10.1109/CVPR.2018.00263. HE Zhang and PATEL V M. Density-aware single image de-raining using a multi-stream dense network[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 695–704. doi: 10.1109/CVPR.2018.00079. FU Xueyang, HUANG Jiabin, DING Xinghao, et al. Clearing the skies: A deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing, 2017, 26(6): 2944–2956. doi: 10.1109/TIP.2017.2691802 肖进胜, 李文昊, 姜红, 等. 基于双域滤波的三维块匹配视频去噪算法[J]. 通信学报, 2015, 36(9): 91–97. doi: 10.11959/j.issn.1000-436x.2015245XIAO Jinsheng, LI Wenhao, JIANG Hong, et al. Three dimensional block-matching video denoising algorithm based on dual-domain filtering[J]. Journal on Communications, 2015, 36(9): 91–97. doi: 10.11959/j.issn.1000-436x.2015245 XIAO Jinsheng, LI Wenhao, LIU Guoxiong, et al. Hierarchical tone mapping based on image colour appearance model[J]. IET Computer Vision, 2014, 8(4): 358–364. doi: 10.1049/iet-cvi.2013.0230 YU Hancheng, LI Zhao, and WANG Haixian. Image denoising using trivariate shrinkage filter in the wavelet domain and joint bilateral filter in the Spatial Domain[J]. IEEE Transactions on Image Processing, 2009, 18(10): 2364–2369. doi: 10.1109/TIP.2009.2026685 HUYNH-THU Q and GHANBARI M. Scope of validity of PSNR in image/video quality assessment[J]. Electronics Letters, 2008, 44(13): 800–801. doi: 10.1049/el:20080522 WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.1109/TIP.2003.819861 -
图(7) / 表(3)
计量
- 文章访问数: 2969
- HTML全文浏览量: 875
- PDF下载量: 178
- 被引次数: 0
下载:
