高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于内外混合图像先验与图像融合的DIP改进降噪模型

徐少平 陈晓军 罗洁 程晓慧 肖楠

徐少平, 陈晓军, 罗洁, 程晓慧, 肖楠. 基于内外混合图像先验与图像融合的DIP改进降噪模型[J]. 电子与信息学报, 2024, 46(1): 299-307. doi: 10.11999/JEIT221580
引用本文: 徐少平, 陈晓军, 罗洁, 程晓慧, 肖楠. 基于内外混合图像先验与图像融合的DIP改进降噪模型[J]. 电子与信息学报, 2024, 46(1): 299-307. doi: 10.11999/JEIT221580
XU Shaoping, CHEN Xiaojun, LUO Jie, CHENG Xiaohui, XIAO Nan. An Improved DIP Denoising Model Based on Internal and External Image Priors and Image Fusion[J]. Journal of Electronics & Information Technology, 2024, 46(1): 299-307. doi: 10.11999/JEIT221580
Citation: XU Shaoping, CHEN Xiaojun, LUO Jie, CHENG Xiaohui, XIAO Nan. An Improved DIP Denoising Model Based on Internal and External Image Priors and Image Fusion[J]. Journal of Electronics & Information Technology, 2024, 46(1): 299-307. doi: 10.11999/JEIT221580

基于内外混合图像先验与图像融合的DIP改进降噪模型

doi: 10.11999/JEIT221580
基金项目: 国家自然科学基金(62162043),江西省研究生创新专项资金(YC2022-s033)
详细信息
    作者简介:

    徐少平:男,博士,教授,博士生导师,研究方向为数字图像处理、机器视觉、虚拟手术模拟

    陈晓军:男,博士生,研究方向为数字图像处理、机器视觉

    罗洁:女,学士,副主任医师, 研究方向为医学图像处理

    程晓慧:女,硕士生,研究方向为数字图像处理、机器视觉

    肖楠:男,硕士生,研究方向为数字图像处理、机器视觉

  • 中图分类号: TN911.73; TP391

An Improved DIP Denoising Model Based on Internal and External Image Priors and Image Fusion

Funds: The National Natural Science Foundation of China(62162043), Jiangxi Postgraduate Innovation Special Fund Project, grant number (YC2022-s033)
  • 摘要: 为提高无监督深度图像先验(DIP)降噪模型的降噪性能,该文提出了一种基于内外混合图像先验与图像融合的DIP改进降噪模型(IDIP),该模型由样本生成和样本融合两个相继执行的模块组成。在样本生成阶段,首先利用2个分别来自内部和外部先验且有代表性的降噪算法(模型)处理噪声图像以产生2张初始降噪图像。基于这2张初始降噪图像,使用空间随机混合器按照各自50%混合比例随机生成足够多的混合图像作为DIP降噪模型的第2目标图像并与第1目标图像(即噪声图像)构成双目标图像。然后,每次使用不同的随机输入和双目标图像,多次执行标准DIP降噪流程生成多张具有互补性的样本图像;在样本融合阶段,首先为了获得更好的随机性和稳定性,随机丢弃50%的样本图像。然后,采用无监督融合网络在样本图像上完成自适应融合,获得的融合图像的图像质量相对参与融合的样本图像得到再次提升,作为最终降噪图像。在人工合成噪声图像上实验表明:IDIP降噪模型较原DIP降噪模型在峰值信噪比评价指标上有约2 dB的提升,且较大幅度超过了其他无监督降噪模型,逼近了有监督降噪模型。而在实际真实噪声图像上,其降噪性能较各对比方法更具鲁棒性。
  • 图  1  IDIP降噪模型的执行流程

    图  2  双目标图像的样本图像生成框架

    图  3  无监督样本图像融合框架

    图  4  各对比方法在SIDD数据集上的降噪视觉效果对比

    表  1  以不同目标图像所获得样本图像的PSNR均值比较(dB)

    目标图像平均值目标图像平均值
    N28.86N+F29.56
    B28.92N+ $t_i\left( {0.25,0.75} \right)$30.12
    F28.94N+ $t_i\left( {0.75,0.25} \right)$30.22
    $t_i\left( {0.5,0.5} \right)$29.54N+ $t_i\left( {0.5,0.5} \right)$30.65
    N+B29.11
    注:其中符号N,B,F和$t_i$分别代表以噪声图像、BM3D预处理图像、FFDNet预处理图像和混合图像单个图像作为目标图像,而N+$t_i\left( {m,n} \right)$代表以噪声图像和混合图像两个图像作为双目标图像,其中$ m $和$ n $分别代表FFDNet和BM3D两张图像混合时所占的比例。
    下载: 导出CSV

    表  2  使用不同样本数量参与融合所获得降噪后图像的PSNR均值比较(dB)

    样本数量平均值样本数量平均值
    230.74630.87
    330.82730.86
    430.87830.82
    530.89
    下载: 导出CSV

    表  3  各对比方法在Set12和BSD68数据集上的平均降噪性能比较(dB)

    数据集BM3DNCSRWNNMDnCNNFFDNetVDNetCsNetDIPN2VRestormerIDIP
    Set1229.0929.2129.3329.6429.7229.5829.6227.9327.4230.2929.92
    BSD6828.5228.4728.5529.1229.0928.2329.3227.2627.0329.4329.47
    下载: 导出CSV
  • [1] 米泽田, 晋洁, 李圆圆, 等. 基于多尺度级联网络的水下图像增强方法[J]. 电子与信息学报, 2022, 44(10): 3353–3362. doi: 10.11999/JEIT220375

    MI Zetian, JIN Jie, LI Yuanyuan, et al. Underwater image enhancement method based on multi-scale cascade network[J]. Journal of Electronics &Information Technology, 2022, 44(10): 3353–3362. doi: 10.11999/JEIT220375
    [2] 张雄, 杨琳琳, 上官宏, 等. 基于生成对抗网络和噪声水平估计的低剂量CT图像降噪方法[J]. 电子与信息学报, 2021, 43(8): 2404–2413. doi: 10.11999/JEIT200591

    ZHANG Xiong, YANG Linlin, SHANGGUAN Hong, et al. A low-dose CT image denoising method based on generative adversarial network and noise level estimation[J]. Journal of Electronics &Information Technology, 2021, 43(8): 2404–2413. doi: 10.11999/JEIT200591
    [3] BIALER O, GARNETT N, and TIRER T. Performance advantages of deep neural networks for angle of arrival estimation[C]. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, UK, 2019: 3907–3911.
    [4] KIM S. Deep recurrent neural networks with layer-wise multi-head attentions for punctuation restoration[C]. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, UK, 2019: 7280–7284.
    [5] ZHANG Kai, ZUO Wangmeng, and ZHANG Lei. FFDNet: Toward a fast and flexible solution for CNN-based image denoising[J]. IEEE Transactions on Image Processing, 2018, 27(9): 4608–4622. doi: 10.1109/TIP.2018.2839891
    [6] ANWAR S and BARNES N. Real image denoising with feature attention[C]. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), 2019: 3155–3164.
    [7] VALSESIA D, FRACASTORO G, and MAGLI E. Deep graph-convolutional image denoising[J]. IEEE Transactions on Image Processing, 2020, 29: 8226–8237. doi: 10.1109/TIP.2020.3013166
    [8] ZAMIR S W, ARORA A, KHAN S, et al. Restormer: Efficient transformer for high-resolution image restoration[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 5718–5729.
    [9] ZHANG Kai, ZUO Wangmeng, CHEN Yunjin, et al. Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3142–3155. doi: 10.1109/TIP.2017.2662206
    [10] LEHTINEN J, MUNKBERG J, HASSELGREN J, et al. Noise2Noise: Learning image restoration without clean data[C]. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 2018: 2971–2980.
    [11] KRULL A, BUCHHOLZ T O, and JUG F. Noise2Void-learning denoising from single noisy images[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 2124–2132.
    [12] HUANG Tao, LI Songjiang, JIA Xu, et al. Neighbor2Neighbor: Self-supervised denoising from single noisy images[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 14776–14785.
    [13] LEMPITSKY V, VEDALDI A, and ULYANOV D. Deep image prior[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 9446–9454.
    [14] ULYANOV D, VEDALDI A, and LEMPITSKY V. Deep image prior[J]. International Journal of Computer Vision, 2020, 128(7): 1867–1888. doi: 10.1007/s11263-020-01303-4
    [15] RONNEBERGER O, FISCHER P, and BROX T. U-Net: Convolutional networks for biomedical image segmentation[C]. Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015: 234–241.
    [16] DABOV K, FOI A, KATKOVNIK V, et al. Image denoising by sparse 3-D transform-domain collaborative filtering[J]. IEEE Transactions on Image Processing, 2007, 16(8): 2080–2095. doi: 10.1109/TIP.2007.901238
    [17] MA Kede, LI Hui, YONG Hongwei, et al. Robust multi-exposure image fusion: A structural patch decomposition approach[J]. IEEE Transactions on Image Processing, 2017, 26(5): 2519–2532. doi: 10.1109/TIP.2017.2671921
    [18] LUO Jingyu, XU Shaoping, and LI Chongxi. A fast denoising fusion network using internal and external priors[J]. Signal, Image and Video Processing, 2021, 15(6): 1275–1283. doi: 10.1007/s11760-021-01858-w
    [19] DONG Weisheng, ZHANG Lei, SHI Guangming, et al. Nonlocally centralized sparse representation for image restoration[J]. IEEE Transactions on Image Processing, 2013, 22(4): 1620–1630. doi: 10.1109/TIP.2012.2235847
    [20] GU Shuhang, ZHANG Lei, ZUO Wangmeng, et al. Weighted nuclear norm minimization with application to image denoising[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, USA, 2014: 2862–2869.
    [21] YUE Zongsheng, YONG Hongwei, ZHAO Qian, et al. Variational denoising network: Toward blind noise modeling and removal[C]. Proceedings of the 33rd Conference on Neural Information Processing Systems, Vancouver, Canada, 2019: 1688–1699.
    [22] CHOI J H, ELGENDY O A, and CHAN S H. Optimal combination of image denoisers[J]. IEEE Transactions on Image Processing, 2019, 28(8): 4016–4031. doi: 10.1109/TIP.2019.2903321
    [23] ABDELHAMED A, LIN S, and BROWN M S. A high-quality denoising dataset for smartphone cameras[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 1692–1700.
  • 加载中
图(4) / 表(3)
计量
  • 文章访问数:  180
  • HTML全文浏览量:  135
  • PDF下载量:  47
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-12-30
  • 修回日期:  2023-06-19
  • 网络出版日期:  2023-06-27
  • 刊出日期:  2024-01-17

目录

    /

    返回文章
    返回