Blind Image Super-resolution Reconstruction via Iterative and Alternative Optimization

CHEN Honggang; LI Ziqiang; ZHANG Yongfei; WANG Zhengyong; QING Linbo; HE Xiaohai

doi:10.11999/JEIT220380

Volume 44 Issue 10

Oct. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(10): 3343-3352

CHEN Honggang, LI Ziqiang, ZHANG Yongfei, WANG Zhengyong, QING Linbo, HE Xiaohai. Blind Image Super-resolution Reconstruction via Iterative and Alternative Optimization[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3343-3352. doi: 10.11999/JEIT220380

Citation:

CHEN Honggang, LI Ziqiang, ZHANG Yongfei, WANG Zhengyong, QING Linbo, HE Xiaohai. Blind Image Super-resolution Reconstruction via Iterative and Alternative Optimization[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3343-3352. doi: 10.11999/JEIT220380

Citation:

CHEN Honggang, LI Ziqiang, ZHANG Yongfei, WANG Zhengyong, QING Linbo, HE Xiaohai. Blind Image Super-resolution Reconstruction via Iterative and Alternative Optimization[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3343-3352. doi: 10.11999/JEIT220380

PDF( 7483 KB)

Blind Image Super-resolution Reconstruction via Iterative and Alternative Optimization

doi: 10.11999/JEIT220380

College of Electronics and Information Engineering, Sichuan University, Chengdu 610065, China

Funds: The National Natural Science Foundation of China (62001316, 61871279), The Natural Science Foundation of Sichuan Province (2022NSFSC0922), The Fundamental Research Foundation for the Central Universities (2021SCU12061)

Received Date: 2022-04-01
Rev Recd Date: 2022-05-21

Available Online: 2022-07-01

Publish Date: 2022-10-19

Abstract

Abstract

Deep convolutional neural network-based image Super-Resolution (SR) methods assume generally that the degradations of Low-Resolution (LR) images are fixed and known (e.g., bicubic downsampling). Thus, they are almost unable to super-resolve images with unknown degradations (e.g., blur kernel and noise level). To address this problem, an iterative and alternative optimization-based blind image SR network is proposed, in which the blur kernel, noise level, and High-Resolution (HR) image are jointly estimated. Specifically, in the proposed method, the image reconstruction network reconstructs an HR image from the given LR image using the estimated blur kernel and noise level as prior knowledge. Correspondingly, the blur kernel and noise level estimators estimate the blur kernel and noise level respectively from the given LR image and the reconstructed HR image. To improve compatibility and promote each other mutually, the blur kernel estimator, noise level estimator, and image reconstruction network are iteratively and alternatively optimized in an end-to-end manner. The proposed network is compared with state-of-the-art methods (i.e., IKC, DASR, MANet, DAN) on commonly used benchmarks (i.e., Set5, Set14, B100, and Urban100) and real-world images. Results show that the proposed method achieves better performance on LR images with unknown degradations. Moreover, the proposed method has advantages in model size or processing efficiency.

FullText(HTML)

References(31)

References

[1]	蔡文郁, 张美燕, 吴岩, 等. 基于循环生成对抗网络的超分辨率重建算法研究[J]. 电子与信息学报, 2022, 44(1): 178–186. doi: 10.11999/JEIT201046 CAI Wenyu, ZHANG Meiyan, WU Yan, et al. Research on cyclic generation countermeasure network based super-resolution image reconstruction algorithm[J]. Journal of Electronics &Information Technology, 2022, 44(1): 178–186. doi: 10.11999/JEIT201046
[2]	ZHANG Xiangjun and WU Xiaolin. Image interpolation by adaptive 2-D autoregressive modeling and soft-decision estimation[J]. IEEE Transactions on Image Processing, 2008, 17(6): 887–896. doi: 10.1109/TIP.2008.924279
[3]	ZHANG Kaibing, GAO Xinbo, TAO Dacheng, et al. Single image super-resolution with non-local means and steering kernel regression[J]. IEEE Transactions on Image Processing, 2012, 21(11): 4544–4556. doi: 10.1109/TIP.2012.2208977
[4]	DONG Chao, LOY C C, HE Kaiming, et al. Learning a deep convolutional network for image super-resolution[C]. Proceedings of the 13th European Conference on Computer Vision, Zurich, Switzerland, 2014: 184–199.
[5]	CHEN Hanting, WANG Yunhe, GUO Tianyu, et al. Pre-trained image processing transformer[C]. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 12299–12310.
[6]	LIANG Jingyun, CAO Jiezhang, SUN Guolei, et al. SwinIR: Image restoration using swin transformer[C]. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops, Montreal, Canada, 2021: 1833–1844.
[7]	ZHANG Kai, LIANG Jingyun, VAN GOOL L, et al. Designing a practical degradation model for deep blind image super-resolution[C]. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 2021: 4791–4800.
[8]	WANG Xintao, XIE Liangbin, DONG Chao, et al. Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data[C]. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 2021: 1905–1914.
[9]	BELL-KLIGLER S, SHOCHER A, and IRANI M. Blind super-resolution kernel estimation using an internal-GAN[C]. Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, Canada, 2019: 284–293.
[10]	LIANG Jingyun, ZHANG Kai, GU Shuhang, et al. Flow-based kernel prior with application to blind super-resolution[C]. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 10601–10610.
[11]	TAO Guangpin, JI Xiaozhong, WANG Wenzhuo, et al. Spectrum-to-kernel translation for accurate blind image super-resolution[C/OL]. Advances in Neural Information Processing Systems, 2021: 34.
[12]	LIANG Jingyun, SUN Guolei, ZHANG Kai, et al. Mutual affine network for spatially variant kernel estimation in blind image super-resolution[C]. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 2021: 4096–4105.
[13]	KIM J, JUNG C, and KIM C. Dual back-projection-based internal learning for blind super-resolution[J]. IEEE Signal Processing Letters, 2020, 27: 1190–1194. doi: 10.1109/LSP.2020.3005043
[14]	WANG Longguang, WANG Yingqian, DONG Xiaoyu, et al. Unsupervised degradation representation learning for blind super-resolution[C]. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 10581–10590.
[15]	GU Jinjin, LU Hannan, ZUO Wangmeng, et al. Blind super-resolution with iterative kernel correction[C]. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 1604–1613.
[16]	LUO Zhengxiong, HUANG Yan, LI Shang, et al. Unfolding the alternating optimization for blind super resolution[C/OL]. Advances in Neural Information Processing Systems, 2020: 5632–5643.
[17]	LUO Zhengxiong, HUANG Yan, LI Shang, et al. End-to-end alternating optimization for blind super resolution[EB/OL]. https://arxiv.org/abs/2105.06878v1, 2021.
[18]	WANG Zhihao, CHEN Jian, and HOI S C H. Deep learning for image super-resolution: A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(10): 3365–3387. doi: 10.1109/TPAMI.2020.2982166
[19]	CHEN Honggang, HE Xiaohai, QING Linbo, et al. Real-world single image super-resolution: A brief review[J]. Information Fusion, 2022, 79: 124–145. doi: 10.1016/j.inffus.2021.09.005
[20]	LIU Anran, LIU Yihao, GU Jinjin, et al. Blind image super-resolution: A survey and beyond[EB/OL]. https://arxiv.org/abs/2107.03055, 2021.
[21]	HUI Zheng, LI Jie, WANG Xiumei, et al. Learning the non-differentiable optimization for blind super-resolution[C]. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 2093–2102.
[22]	CHEN Haoyu, GU Jinjin, and ZHANG Zhi. Attention in attention network for image super-resolution[EB/OL]. https://arxiv.org/abs/2104.09497v3, 2021.
[23]	GUO Shi, YAN Zifei, ZHANG Kai, et al. Toward convolutional blind denoising of real photographs[C]. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 1712–1722.
[24]	AGUSTSSON E and TIMOFTE R. NTIRE 2017 challenge on single image super-resolution: Dataset and study[C]. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, USA, 2017: 126–135.
[25]	TIMOFTE R, AGUSTSSON E, VAN GOOL L, et al. NTIRE 2017 challenge on single image super-resolution: Methods and results[C]. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, USA, 2017: 114–125.
[26]	KINGMA D P and BA J. ADAM: A method for stochastic optimization[C]. 3rd International Conference on Learning Representations, San Diego, USA, 2014.
[27]	BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]. British Machine Vision Conference, Surrey, United Kingdom, 2012.
[28]	ZEYDE R, ELAD M, and PROTTER M. On single image scale-up using sparse-representations[C]. 7th International Conference on Curves and Surfaces, Avignon, France, 2010: 711–730.
[29]	MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]. Proceedings Eighth IEEE International Conference on Computer Vision, Vancouver, Canada, 2001: 416–423.
[30]	HUANG J B, SINGH A, and AHUJA N. Single image super-resolution from transformed self-exemplars[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 5197–5206.
[31]	ZHANG Kai, ZUO Wangmeng, CHEN Yunjin, et al. Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3142–3155. doi: 10.1109/TIP.2017.2662206