An Improved DIP Denoising Model Based on Internal and External Image Priors and Image Fusion

XU Shaoping; CHEN Xiaojun; LUO Jie; CHENG Xiaohui; XIAO Nan

doi:10.11999/JEIT221580

Volume 46 Issue 1

Jan. 2024

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2024 > 46(1): 299-307

XU Shaoping, CHEN Xiaojun, LUO Jie, CHENG Xiaohui, XIAO Nan. An Improved DIP Denoising Model Based on Internal and External Image Priors and Image Fusion[J]. Journal of Electronics & Information Technology, 2024, 46(1): 299-307. doi: 10.11999/JEIT221580

Citation:

XU Shaoping, CHEN Xiaojun, LUO Jie, CHENG Xiaohui, XIAO Nan. An Improved DIP Denoising Model Based on Internal and External Image Priors and Image Fusion[J]. Journal of Electronics & Information Technology, 2024, 46(1): 299-307. doi: 10.11999/JEIT221580

Citation:

PDF( 4643 KB)

An Improved DIP Denoising Model Based on Internal and External Image Priors and Image Fusion

doi: 10.11999/JEIT221580

1.
School of Mathematics and Computer Sciences, Nanchang University, Nanchang 330031, China
2.
Infectious Disease Hospital Affiliated to Nanchang University , Nanchang 330006, China

Funds: The National Natural Science Foundation of China(62162043), Jiangxi Postgraduate Innovation Special Fund Project, grant number (YC2022-s033)

Received Date: 2022-12-30
Rev Recd Date: 2023-06-19

Available Online: 2023-06-27

Publish Date: 2024-01-17

Abstract

Abstract

To enhance the denoising performance of an unsupervised Deep Image Prior (DIP) model, an improved approach known as the Improved Deep Image Prior (IDIP) is proposed, which comprises sample generation and sample fusion modules, and leverages a prior hybrid image that combines internal and external factors, along with image fusion techniques. In the sample generation module, two representative denoising models are utilized, which capture internal and external priors and process the noisy image to produce two initial denoised images. Subsequently, a spatially random mixer is implemented on these initial denoised images to generate a sufficient number of mixed images. These mixed images, along with the noisy image, form dual-target images with a 50% mixing ratio. Furthermore, executing the standard DIP denoising process multiple times with different random inputs and dual-target images generates a set of diverse sample images with complementary characteristics. In the sample fusion module, to enhance randomness and stability, 50% of the sample images are randomly discarded using dropout. Next, an unsupervised fusion network is used, which performs adaptive fusion on the remaining sample images. The resulting fused image exhibits improved image quality compared to the individual sample images and serves as the final denoised output. The experimental results on artificially generated noisy images reveal that the IDIP model is effective, with an improvement of approximately 2 dB in terms of Peak Signal-to-Noise Ratio (PSNR) compared to the original DIP model. Moreover, the IDIP model outperforms other unsupervised denoising models by a significant margin and approaches the performance level of supervised denoising models. When evaluated on real-world noisy images, the IDIP model exhibits superior denoising performance to the compared methods, thus verifying its robustness.
- Image denoising,
- Deep image prior,
- Boosting performance,
- Internal and external image priors,
- Unsupervised fusion

FullText(HTML)

References(23)

References

[1]	米泽田, 晋洁, 李圆圆, 等. 基于多尺度级联网络的水下图像增强方法[J]. 电子与信息学报, 2022, 44(10): 3353–3362. doi: 10.11999/JEIT220375 MI Zetian, JIN Jie, LI Yuanyuan, et al. Underwater image enhancement method based on multi-scale cascade network[J]. Journal of Electronics &Information Technology, 2022, 44(10): 3353–3362. doi: 10.11999/JEIT220375
[2]	张雄, 杨琳琳, 上官宏, 等. 基于生成对抗网络和噪声水平估计的低剂量CT图像降噪方法[J]. 电子与信息学报, 2021, 43(8): 2404–2413. doi: 10.11999/JEIT200591 ZHANG Xiong, YANG Linlin, SHANGGUAN Hong, et al. A low-dose CT image denoising method based on generative adversarial network and noise level estimation[J]. Journal of Electronics &Information Technology, 2021, 43(8): 2404–2413. doi: 10.11999/JEIT200591
[3]	BIALER O, GARNETT N, and TIRER T. Performance advantages of deep neural networks for angle of arrival estimation[C]. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, UK, 2019: 3907–3911.
[4]	KIM S. Deep recurrent neural networks with layer-wise multi-head attentions for punctuation restoration[C]. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, UK, 2019: 7280–7284.
[5]	ZHANG Kai, ZUO Wangmeng, and ZHANG Lei. FFDNet: Toward a fast and flexible solution for CNN-based image denoising[J]. IEEE Transactions on Image Processing, 2018, 27(9): 4608–4622. doi: 10.1109/TIP.2018.2839891
[6]	ANWAR S and BARNES N. Real image denoising with feature attention[C]. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), 2019: 3155–3164.
[7]	VALSESIA D, FRACASTORO G, and MAGLI E. Deep graph-convolutional image denoising[J]. IEEE Transactions on Image Processing, 2020, 29: 8226–8237. doi: 10.1109/TIP.2020.3013166
[8]	ZAMIR S W, ARORA A, KHAN S, et al. Restormer: Efficient transformer for high-resolution image restoration[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 5718–5729.
[9]	ZHANG Kai, ZUO Wangmeng, CHEN Yunjin, et al. Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3142–3155. doi: 10.1109/TIP.2017.2662206
[10]	LEHTINEN J, MUNKBERG J, HASSELGREN J, et al. Noise2Noise: Learning image restoration without clean data[C]. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 2018: 2971–2980.
[11]	KRULL A, BUCHHOLZ T O, and JUG F. Noise2Void-learning denoising from single noisy images[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 2124–2132.
[12]	HUANG Tao, LI Songjiang, JIA Xu, et al. Neighbor2Neighbor: Self-supervised denoising from single noisy images[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 14776–14785.
[13]	LEMPITSKY V, VEDALDI A, and ULYANOV D. Deep image prior[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 9446–9454.
[14]	ULYANOV D, VEDALDI A, and LEMPITSKY V. Deep image prior[J]. International Journal of Computer Vision, 2020, 128(7): 1867–1888. doi: 10.1007/s11263-020-01303-4
[15]	RONNEBERGER O, FISCHER P, and BROX T. U-Net: Convolutional networks for biomedical image segmentation[C]. Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015: 234–241.
[16]	DABOV K, FOI A, KATKOVNIK V, et al. Image denoising by sparse 3-D transform-domain collaborative filtering[J]. IEEE Transactions on Image Processing, 2007, 16(8): 2080–2095. doi: 10.1109/TIP.2007.901238
[17]	MA Kede, LI Hui, YONG Hongwei, et al. Robust multi-exposure image fusion: A structural patch decomposition approach[J]. IEEE Transactions on Image Processing, 2017, 26(5): 2519–2532. doi: 10.1109/TIP.2017.2671921
[18]	LUO Jingyu, XU Shaoping, and LI Chongxi. A fast denoising fusion network using internal and external priors[J]. Signal, Image and Video Processing, 2021, 15(6): 1275–1283. doi: 10.1007/s11760-021-01858-w
[19]	DONG Weisheng, ZHANG Lei, SHI Guangming, et al. Nonlocally centralized sparse representation for image restoration[J]. IEEE Transactions on Image Processing, 2013, 22(4): 1620–1630. doi: 10.1109/TIP.2012.2235847
[20]	GU Shuhang, ZHANG Lei, ZUO Wangmeng, et al. Weighted nuclear norm minimization with application to image denoising[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, USA, 2014: 2862–2869.
[21]	YUE Zongsheng, YONG Hongwei, ZHAO Qian, et al. Variational denoising network: Toward blind noise modeling and removal[C]. Proceedings of the 33rd Conference on Neural Information Processing Systems, Vancouver, Canada, 2019: 1688–1699.
[22]	CHOI J H, ELGENDY O A, and CHAN S H. Optimal combination of image denoisers[J]. IEEE Transactions on Image Processing, 2019, 28(8): 4016–4031. doi: 10.1109/TIP.2019.2903321
[23]	ABDELHAMED A, LIN S, and BROWN M S. A high-quality denoising dataset for smartphone cameras[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 1692–1700.