Adaptive Image Interpolation Algorithm and Acceleration Engine Co-Design

YAN Xinkai; DING Sheng

doi:10.11999/JEIT221503

Volume 45 Issue 9

Sep. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 > 45(9): 3284-3294

YAN Xinkai, DING Sheng. Adaptive Image Interpolation Algorithm and Acceleration Engine Co-Design[J]. Journal of Electronics & Information Technology, 2023, 45(9): 3284-3294. doi: 10.11999/JEIT221503

Citation:

YAN Xinkai, DING Sheng. Adaptive Image Interpolation Algorithm and Acceleration Engine Co-Design[J]. Journal of Electronics & Information Technology, 2023, 45(9): 3284-3294. doi: 10.11999/JEIT221503

YAN Xinkai, DING Sheng. Adaptive Image Interpolation Algorithm and Acceleration Engine Co-Design[J]. Journal of Electronics & Information Technology, 2023, 45(9): 3284-3294. doi: 10.11999/JEIT221503

Citation:

PDF( 3719 KB)

Adaptive Image Interpolation Algorithm and Acceleration Engine Co-Design

doi: 10.11999/JEIT221503

YAN Xinkai^{1, 2
,
,},
DING Sheng²

1.
Zhejiang University, Hangzhou 310015, China
2.
Jiangsu Key Laboratory Of Asic Design (Wuxi), Wuxi 214153, China

Funds: The Natural Science Foundation of the Jiangsu Higher Education Institutions of China (19KJB510027), Jiangsu “333” Scientific Research Project (BRA2020318), The Development Fundation of Jiangsu Key Laboratory of Asic Design (2020KLOP005)

Received Date: 2022-12-02
Rev Recd Date: 2023-04-12

Available Online: 2023-04-19

Publish Date: 2023-09-27

Abstract

Abstract

In order to improve the super-resolution reconstruction effect of the high-definition color image, a new adaptive image interpolation algorithm based on edge contrast is proposed, which chooses adaptively the coefficients of Lanczos interpolation by edge contrast detection and receptive fields with different scales. Adaptability and diverse receptive fields can further improve the quality of image magnification. Compared with the bilinear interpolation algorithm, the Peak Signal to Noise Ratio (PSNR), Structural SIMilarity (SSIM) and Learned Perceptual Image Patch Similarity (LPIPS) are improved by 1.1 dB, 0.025, 0.051, respectively. Compared with the bicubic interpolation algorithm, the PSNR, SSIM and LPIPS are improved by 0.34 dB, 0.01, 0.033, respectively. Moreover, in order to reduce the hardware resources and improve the storage efficiency, a high parallelized and high efficiency accelerated architecture is proposed. A 2-level data reuse and coefficients pulsation mechanism are employed to improve the computation-memory access ratio greatly. The synthesis result of the acceleration engine in the 16nm process library can reach the 2 GHz clock frequency. The operating frequency of FPGA project deployed in Xilinx Zynq Ultra scale+ xczu15eg can reach up to 200 MHz as well, which means that the algorithm can adapt to the frame rate (fps) up to 60.
- Interpolation algorithm,
- Adaptive,
- Parallelism,
- Energy efficiency,
- Acceleration engine

FullText(HTML)

References(26)

References

[1]	MEIJERING E H W, ZUIDERVELD K J, and VIERGEVER M A. Image reconstruction by convolution with symmetrical piecewise nth-order polynomial kernels[J]. IEEE Transactions on Image Processing, 1999, 8(2): 192–201. doi: 10.1109/83.743854
[2]	KEYS R G. Cubic convolution interpolation for digital image processing[J]. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1981, 29(6): 1153–1160. doi: 10.1109/TASSP.1981.1163711
[3]	KWOK W and SUN H. Multi-directional interpolation for spatial error concealment[J]. IEEE Transactions on Consumer Electronics, 1993, 39(3): 455–460. doi: 10.1109/30.234620
[4]	LI Xin and ORCHARD M T. New edge-directed interpolation[J]. IEEE Transactions on Image Processing, 2001, 10(10): 1521–1527. doi: 10.1109/83.951537
[5]	CHEN Meijuan, HUANG C H, and LEE W L. A fast edge-oriented algorithm for image interpolation[J]. Image and Vision Computing, 2005, 23(9): 791–798. doi: 10.1016/j.imavis.2005.05.005
[6]	ZHANG Xiangjun and WU Xiaolin. Image interpolation by adaptive 2-D autoregressive modeling and soft-decision estimation[J]. IEEE Transactions on Image Processing, 2008, 28(6): 887–896. doi: 10.1109/TIP.2008.924279
[7]	JAKHETIYA V, KUMAR A, and TIWARI A K. Image interpolation by adaptive 2-D autoregressive modeling[C]. Proceedings of SPIE 7546, Second International Conference on Digital Image Processing, Singapore, 2010.
[8]	LIU Yiwei, JIANG Zhuqing, WANG Yibo, et al. Single-frame reconstruction for improvement of off-axis digital holographic imaging based on image interpolation[J]. Optics Letters, 2020, 45(24): 6623–6626. doi: 10.1364/OL.405578
[9]	WANG Qiang, TANG Xiaoou, and SHUM H. Patch based blind image super resolution[C]. Proceedings of the Tenth IEEE International Conference on Computer Vision, Beijing, China, 2005: 709–716.
[10]	CHAN T M, ZHANG Junping, PU Jian, et al. Neighbor embedding based super-resolution algorithm through edge detection and feature selection[J]. Pattern Recognition Letters, 2009, 30(5): 494–502. doi: 10.1016/j.patrec.2008.11.008
[11]	GAO Xinbo, ZHANG Kaibing, TAO Dacheng, et al. Joint learning for single-image super-resolution via a coupled constraint[J]. IEEE Transactions on Image Processing, 2012, 21(2): 469–480. doi: 10.1109/TIP.2011.2161482
[12]	JI Jiahuan, ZHONG Baojiang, and MA Kaikuang. Image interpolation using multi-scale attention-aware inception network[J]. IEEE Transactions on Image Processing, 2020, 29: 9413–9428. doi: 10.1109/TIP.2020.3026632
[13]	NIU Ben, WEN Weilei, REN Wenqi, et al. Single image super-resolution via a holistic attention network[C]. Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK, 2020: 191–207.
[14]	WEI Pengxu, XIE Ziwei, LU Hannan, et al. Component divide-and-conquer for real-world image super-resolution[C]. Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK, 2020: 101–117.
[15]	DENG Xin, ZHANG Yutong, XU Mai, et al. Deep coupled feedback network for joint exposure fusion and image super-resolution[J]. IEEE Transactions on Image Processing, 2021, 30: 3098–3112. doi: 10.1109/TIP.2021.3058764
[16]	LIN Yuting, LIU Wei, CAI Xiaowen, et al. A CNN-based quality model for image interpolation[C]. Proceedings of 2020 Cross Strait Radio Science & Wireless Technology Conference, Fuzhou, China, 2020: 1–3.
[17]	AMD. AMD FidelityFX super resolution (FSR): Changing the game in just 4 months[EB/OL]. https://www.amd.com/zh-hans/technologies/fidelityfx-super-resolution, 2021.
[18]	Andrew Burnes, nvidia-image-scaler-dlss-rtx-november-2021-updates[EB/OL]. https://www.nvidia.com/en-us/geforce/news/gfecnt/202111/nvidia-image-scaler-dlss-rtx-november-2021-updates/, 2021.
[19]	KHALEDYAN D, AMIRANY A, JAFARI K, et al. Low-cost implementation of bilinear and bicubic image interpolation for real-time image super-resolution[C]. Proceedings of 2020 IEEE Global Humanitarian Technology Conference, Seattle, USA, 2020: 1–5.
[20]	王康, 杨瑞祺, 杨依忠, 等. 基于二阶牛顿插值的图像自适应缩放设计及实现[J]. 计算机应用与软件, 2020, 37(9): 126–132,138. doi: 10.3969/j.issn.1000-386x.2020.09.021 WANG Kang, YANG Ruiqi, YANG Yizhong, et al. Design and implementation of image adaptive scaling based on second order newton interpolation[J]. Computer Applications and Software, 2020, 37(9): 126–132,138. doi: 10.3969/j.issn.1000-386x.2020.09.021
[21]	BOUKHTACHE S, BLAYSAT B, GRÉDIAC M, et al. FPGA-based architecture for bi-cubic interpolation: The best trade-off between precision and hardware resource consumption[J]. Journal of Real-Time Image Processing, 2021, 18(3): 901–911. doi: 10.1007/s11554-020-01035-1
[22]	SHEN H Y, LEE Y C, TONG T W, et al. 4.7 A 91mW 90fps super-resolution processor for full HD images[C]. Proceedings of 2021 IEEE International Solid- State Circuits Conference, San Francisco, USA, 2021: 66–68.
[23]	陆志芳, 钟宝江. 基于预测梯度的图像插值算法[J]. 自动化学报, 2018, 44(6): 1072–1085. doi: 10.16383/j.aas.2017.c160793 LU Zhifang and ZHONG Baojiang. Image interpolation with predicted gradients[J]. Acta Automatica Sinica, 2018, 44(6): 1072–1085. doi: 10.16383/j.aas.2017.c160793
[24]	ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]. Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 586–595.
[25]	吴世豪, 罗小华, 张建炜, 等. 基于FPGA的新边缘指导插值算法硬件实现[J]. 浙江大学学报:工学版, 2018, 52(11): 2226–2232. doi: 10.3785/j.issn.1008-973X.2018.11.022 WU Shihao, LUO Xiaohua, ZHANG Jianwei, et al. FPGA-based hardware implementation of new edge-directed interpolation algorithm[J]. Journal of Zhejiang University:Engineering Science, 2018, 52(11): 2226–2232. doi: 10.3785/j.issn.1008-973X.2018.11.022
[26]	LEE J, SHIN D, LEE J, et al. A full HD 60 fps CNN super resolution processor with selective caching based layer fusion for mobile devices[C]. Proceedings of 2019 Symposium on VLSI Circuits. Kyoto, Japan, 2019: C302–C303.