Super-Resolution Reconstruction of Deep Residual Network with Multi-Level Skip Connections

Xiaoqiang ZHAO; Zhaoyang SONG

doi:10.11999/JEIT190036

Volume 41 Issue 10

Oct. 2019

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2019 > 41(10): 2501-2508

Xiaoqiang ZHAO, Zhaoyang SONG. Super-Resolution Reconstruction of Deep Residual Network with Multi-Level Skip Connections[J]. Journal of Electronics & Information Technology, 2019, 41(10): 2501-2508. doi: 10.11999/JEIT190036

Citation:

Xiaoqiang ZHAO, Zhaoyang SONG. Super-Resolution Reconstruction of Deep Residual Network with Multi-Level Skip Connections[J]. Journal of Electronics & Information Technology, 2019, 41(10): 2501-2508. doi: 10.11999/JEIT190036

Citation:

PDF( 1321 KB)

Super-Resolution Reconstruction of Deep Residual Network with Multi-Level Skip Connections

doi: 10.11999/JEIT190036

Xiaoqiang ZHAO^{1, 2, 3
,
,},
Zhaoyang SONG¹

1.
College of Electrical Engineering and Information Engineering, Lanzhou University of Technology, Lanzhou 730050, China
2.
Key Laboratory of Gansu Advanced Control for Industrial Processes, Lanzhou University of Technology, Lanzhou 730050, China
3.
National Experimental Teaching Center of Electrical and Control Engineering, Lanzhou University of Technology, Lanzhou 730050, China

Funds: The National Natural Science Foundation of China (61763029, 61873116)

Received Date: 2019-01-15
Rev Recd Date: 2019-06-30

Available Online: 2019-07-19

Publish Date: 2019-10-01

Abstract

Abstract

The Fast Super-Resolution Convolutional Neural Network algorithm (FSRCNN) is difficult to extract deep image information due to the small number of convolution layers and the correlation lack between the feature information of adjacent convolutional layers. To solve this problem, a deep residual network super-resolution reconstruction method with multi-level skip connections is proposed. Firstly, a residual block with multi-level skip connections is designed to solve the problem that the characteristic information of adjacent convolutional layers lacks relevance. A deep residual network with multi-level skip connections is constructed on the basis of the residual block. Then, the deep residual network connected to the multi-level skip is trained by using the adaptive gradient rate strategy of Stochastic Gradient Descent (SGD) method and the network super-resolution reconstruction model is obtained. Finally, the low-resolution image is input into the deep residual network super-resolution reconstruction model with the multi-level skip connections, and the residual eigenvalue is obtained by the residual block connected the multi-level skip connections. The residual eigenvalue and the low resolution image are combined and converted into a high resolution image. The proposed method is compared with the bicubic, A+, SRCNN, FSRCNN and ESPCN algorithms in the Set5 and Set14 test sets. The proposed method is superior to other comparison algorithms in terms of visual effects and evaluation index values.

FullText(HTML)

References(29)

References

THORNTON M W, ATKINSON P M, and HOLLAND D A. Sub-pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super-resolution pixel-swapping[J]. International Journal of Remote Sensing, 2006, 27(3): 473–491. doi: 10.1080/01431160500207088

PELED S and YESHURUN Y. Superresolution in MRI: Application to human white matter fiber tract visualization by diffusion tensor imaging[J]. Magnetic Resonance in Medicine, 2001, 45(1): 29–35. doi: 10.1002/1522-2594(200101)45

ZOU W W W and YUEN P C. Very low resolution face recognition problem[J]. IEEE Transactions on Image Processing, 2012, 21(1): 327–340. doi: 10.1109/TIP.2011.2162423

LU Huimin, LI Yujie, CHEN Min, et al. Brain intelligence: Go beyond artificial intelligence[J]. Mobile Networks and Applications, 2018, 23(2): 368–375. doi: 10.1007/s11036-017-0932-8

KOCH M. Artificial intelligence is becoming natural[J]. Cell, 2018, 173(3): 531–533. doi: 10.1016/j.cell.2018.04.007

LEO M, MEDIONI G, TRIVEDI M, et al. Computer vision for assistive technologies[J]. Computer Vision and Image Understanding, 2017, 154: 1–15. doi: 10.1016/j.cviu.2016.09.001

ZHU Hong, TANG Xinming, XIE Junfeng, et al. Spatio-temporal super-resolution reconstruction of remote-sensing images based on adaptive multi-scale detail enhancement[J]. Sensors, 2018, 18(2): 498. doi: 10.3390/s18020498

SHI Jun, LIU Qingping, WANG Chaofeng, et al. Super-resolution reconstruction of MR image with a novel residual learning network algorithm[J]. Physics in Medicine & Biology, 2018, 63(8): 085011. doi: 10.1088/1361-6560/aab9e9

SU Heng, ZHOU Jie, and ZHANG Zhihao. Survey of super-resolution image reconstruction methods[J]. Acta Automatica Sinica, 2013, 39(8): 1202–1213. doi: 10.3724/SP.J.1004.2013.01202

GRIBBON K T and BAILEY D G. A novel approach to real-time bilinear interpolation[C]. The 2nd IEEE International Workshop on Electronic Design, Test and Applications, Perth, Australia, 2004: 126–131. doi: 10.1109/DELTA.2004.10055.

FRITSCH F N and CARLSON R E. Monotone piecewise cubic interpolation[J]. SIAM Journal on Numerical Analysis, 1980, 17(2): 238–246. doi: 10.1137/0717021

STARK H and OSKOUI P. High-resolution image recovery from image-plane arrays, using convex projections[J]. Journal of the Optical Society of America A, 1989, 6(11): 1715–1726. doi: 10.1364/JOSAA.6.001715

PATANAVIJIT V and JITAPUNKUL S. An iterative super-resolution reconstruction of image sequences using fast affine block-based registration with BTV regularization[C]. Proceedings of 2006 IEEE Asia Pacific Conference on Circuits and Systems, Singapore, 2006: 1717–1720. doi: 10.1109/APCCAS.2006.342128.

ZHOU Fei, YANG Wenming, and LIAO Qingmin. Interpolation-based image super-resolution using multisurface fitting[J]. IEEE Transactions on Image Processing, 2012, 21(7): 3312–3318. doi: 10.1109/TIP.2012.2189576

LIN Zhouchen and SHUM H Y. Fundamental limits of reconstruction-based superresolution algorithms under local translation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(1): 83–97. doi: 10.1109/TPAMI.2004.1261081

YOUNG T, HAZARIKA D, PORIA S, et al. Recent trends in deep learning based natural language processing[J]. IEEE Computational Intelligence Magazine, 2018, 13(3): 55–75. doi: 10.1109/MCI.2018.2840738

KERMANY D S, GOLDBAUM M, CAI Wenjia, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning[J]. Cell, 2018, 172(5): 1122–1131.e9. doi: 10.1016/j.cell.2018.02.010

DONG Chao, LOY C C, HE Kaiming, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295–307. doi: 10.1109/TPAMI.2015.2439281

DONG Chao, LOY C C, and TANG Xiaoou. Accelerating the super-resolution convolutional neural network[C]. The 14th European Conference on Computer Vision, Amsterdam, The Netherlands, 2016: 391–407. doi: 10.1007/978-3-319-46475-6_25.

KIM J, LEE J K, and LEE K M. Deeply-recursive convolutional network for image super-resolution[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 1637–1645. doi: 10.1109/CVPR.2016.181.

SHI Wenzhe, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 1874–1883. doi: 10.1109/CVPR.2016.207.

LEDIG C, THEIS L, HUSZÁR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 4681–4690. doi: 10.1109/CVPR.2017.19.

HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. doi: 10.1109/CVPR.2016.90.

KIM J, LEE J K, and LEE K M. Accurate image super-resolution using very deep convolutional networks[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 1646–1654. doi: 10.1109/CVPR.2016.182.

YUAN Fei, HUANG Lianfen, and YAO Yan. An improved PSNR algorithm for objective video quality evaluation[C]. 2007 Chinese Control Conference, Hunan, China, 2007: 376–380. doi: 10.1109/CHICC.2006.4347144.

WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.1109/TIP.2003.819861

GAO Shengkui and GRUEV V. Bilinear and bicubic interpolation methods for division of focal plane polarimeters[J]. Optics Express, 2011, 19(27): 26161–26173. doi: 10.1364/OE.19.026161

TIMOFTE R, DE SMET V, and VAN GOOL L. A+: Adjusted anchored neighborhood regression for fast super-resolution[C]. The 12th Asian Conference on Computer Vision, Singapore, 2014: 111–126. doi: 10.1007/978-3-319-16817-3_8.

ZEYDE R, ELAD M, and PROTTER M. On single image scale-up using sparse-representations[C]. The 7th International Conference on Curves and Surfaces, Avignon, France, 2010: 711–730. doi: 10.1007/978-3-642-27413-8_47.

Relative Articles

Supplements(0)

Cited By

Proportional views