Pseudo Supervised Attention Short-term Memory and Multi-Scale Deartifacting Network Based on Image Block Compressed Sensing

LI Junhui; HOU Xingsong

doi:10.11999/JEIT231069

Volume 46 Issue 2

Feb. 2024

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2024 > 46(2): 472-480

LI Junhui, HOU Xingsong. Pseudo Supervised Attention Short-term Memory and Multi-Scale Deartifacting Network Based on Image Block Compressed Sensing[J]. Journal of Electronics & Information Technology, 2024, 46(2): 472-480. doi: 10.11999/JEIT231069

Citation:

LI Junhui, HOU Xingsong. Pseudo Supervised Attention Short-term Memory and Multi-Scale Deartifacting Network Based on Image Block Compressed Sensing[J]. Journal of Electronics & Information Technology, 2024, 46(2): 472-480. doi: 10.11999/JEIT231069

Citation:

PDF( 8821 KB)

Pseudo Supervised Attention Short-term Memory and Multi-Scale Deartifacting Network Based on Image Block Compressed Sensing

doi: 10.11999/JEIT231069

LI Junhui,
HOU Xingsong^,

School of Information and Communication Engineering, Xi’an Jiaotong University, Xi’an 710049, China

Funds: The National Natural Science Foundation of China (62272376, 61872286), Key R&D Program of Shaanxi Province, China (202DLGY04-05, S2021-YF-YBSF-0094)

Received Date: 2023-10-07
Rev Recd Date: 2024-01-17

Available Online: 2024-01-20

Publish Date: 2024-02-29

Abstract

Abstract

Deep unfolding network based Block Compressed Sensing (BCS) methods typically remove some signal and retain certain block artifacts simultaneously during iterative deartifacting, which is unfavorable for signal recovery. To enhance reconstruction performance, based on Learned Denoising Iterative Thresholding (LDIT) algorithm. Pseudo Supervised Attention Short-term Memory and Multi-scale Deartifacting (PSASM-MD) based image BCS, is proposed in this paper. Initially, in each iteration, each image block is denoised separately in parallel using residual networks before being concatenated. Subsequently, in conjunction with the Pseudo-Supervised Attention Module (PSAM), Multi-Scale Deartifacting Network (MSD-Net) is used to perform feature extraction on the concatenated images, enabling more efficient removal of block artifacts and improving the reconstruction performance. In this case, PSAM is utilized to extract useful signal components from the residuals containing block artifacts, transfer the short-term memory to the subsequent iteration to minimize the removal of useful signals. Experimental results demonstrate that this approach outperforms existing state-of-the-art BCS methods both in subjective visual perception and objective evaluation metrics.
- Block Compressed Sensing (BCS),
- Short-term memory,
- Image deartifacting,
- Deep unfolding network

FullText(HTML)

References(25)

References

[1]	HOU Hao, SHAO Yuchen, GENG Yang, et al. PNCS: Pixel-level non-local method based compressed sensing undersampled MRI image reconstruction[J]. IEEE Access, 2023, 11: 42389–42402. doi: 10.1109/ACCESS.2023.3270900.
[2]	王洋, 杨孟宇, 赵首博. 基于自适应分块的高光谱图像压缩感知重构方法[J]. 电子与信息学报, 2023, 45(7): 2605–2613. doi: 10.11999/JEIT220738. WANG Yang, YANG Mengyu, and ZHAO Shoubo. Compressed sensing reconstruction of hyperspectral images based on adaptive blocking[J]. Journal of Electronics & Information Technology, 2023, 45(7): 2605–2613. doi: 10.11999/JEIT220738.
[3]	LIU Yang, YUAN Xin, SUO Jinli, et al. Rank minimization for snapshot compressive imaging[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(12): 2990–3006. doi: 10.1109/TPAMI.2018.2873587.
[4]	TROPP J A and GILBERT A C. Signal recovery from random measurements via orthogonal matching pursuit[J]. IEEE Transactions on Information Theory, 2007, 53(12): 4655–4666. doi: 10.1109/TIT.2007.909108.
[5]	BECK A and TEBOULLE M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems[J]. SIAM Journal on Imaging Sciences, 2009, 2(1): 183–202. doi: 10.1137/080716542.
[6]	METZLER C A, MOUSAVI A, and BARANIUK R. Learned D-AMP: Principled neural network based compressive image recovery[C]. The 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 1770–1781. doi: 10.5555/3294771.3294940.
[7]	MOUSAVI A, PATEL A B, and BARANIUK R G. A deep learning approach to structured signal recovery[C]. The 53rd Annual Allerton Conference on Communication, Control, and Computing, Monticello, USA, 2015: 1336–1343. doi: 10.1109/ALLERTON.2015.7447163.
[8]	KULKARNI K, LOHIT S, TURAGA P, et al. ReconNet: Non-iterative reconstruction of images from compressively sensed measurements[C]. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 449–458. doi: 10.1109/CVPR.2016.55.
[9]	GAN Hongping, SHEN Minghe, HUA Yi, et al. From patch to pixel: A transformer-based hierarchical framework for compressive image sensing[J]. IEEE Transactions on Computational Imaging, 2023, 9: 133–146. doi: 10.1109/TCI.2023.3244396.
[10]	LIU Ze, LIN Yutong, GAO Yue, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C]. IEEE International Conference on Computer Vision, Montreal, Canada, 2021: 9992–10002. doi: 10.1109/ICCV48922.2021.00986.
[11]	ZHANG Jian, CHEN Bin, XIONG Ruiqin, et al. Physics-inspired compressive sensing: Beyond deep unrolling[J]. IEEE Signal Processing Magazine, 2023, 40(1): 58–72. doi: 10.1109/MSP.2022.3208394.
[12]	ZHANG Jian and GHANEM B. ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing[C]. IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 1828–1837. doi: 10.1109/CVPR.2018.00196.
[13]	ZHANG Jian, ZHAO Chen, and GAO Wen. Optimization-inspired compact deep compressive sensing[J]. IEEE Journal of Selected Topics in Signal Processing. 2020, 14(4): 765–774. doi: 10.1109/JSTSP.2020.2977507.
[14]	ZHANG Zhonghao, LIU Yipeng, LIU Jiani, et al. AMP-Net: Denoising-based deep unfolding for compressive image sensing[J]. IEEE Transactions on Image Processing, 2021, 30: 1487–1500. doi: 10.1109/TIP.2020.3044472.
[15]	CHEN Chang, XIONG Zhiwei, TIAN Xinmei, et al. Real-world image denoising with deep boosting[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(12): 3071–3087. doi: 10.1109/TPAMI.2019.2921548.
[16]	MA Jiayi, PENG Chengli, TIAN Xin, et al. DBDnet: A deep boosting strategy for image denoising[J]. IEEE Transactions on Multimedia, 2022, 24: 3157–3168. doi: 10.1109/TMM.2021.3094058.
[17]	SHEN Minghe, GAN Hongping, NING Chao, et al. TransCS: A transformer-based hybrid architecture for image compressed sensing[J]. IEEE Transactions on Image Processing, 2022, 31: 6991–7005. doi: 10.1109/TIP.2022.3217365.
[18]	CUI Wenxue, LIU Shaohui, JIANG Feng, et al. Image compressed sensing using non-local neural network[J]. IEEE Transactions on Multimedia, 2023, 25: 816–830. doi: 10.1109/TMM.2021.3132489.
[19]	ZHANG Kai, ZUO Wangmeng, CHEN Yunjin, et al. Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3142–3155. doi: 10.1109/TIP.2017.2662206.
[20]	RONNEBERGER O, FISCHER P, and BROX T. U-Net: Convolutional networks for biomedical image segmentation[C]. 18th International Conference, Munich, Germany, 2015: 234–241. doi: 10.1007/978-3-319-24574-4_28.
[21]	WANG Huake, LI Ziang, and HOU Xingsong. Versatile denoising-based approximate message passing for compressive sensing[J]. IEEE Transactions on Image Processing, 2023, 32: 2761–2775. doi: 10.1109/TIP.2023.3274967.
[22]	LIU Pengju, ZHANG Hongzhi, ZHANG Kai, et al. Multi-level wavelet-CNN for image restoration[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, USA, 2018: 886. doi: 10.1109/CVPRW.2018.00121.
[23]	MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]. 8th IEEE International Conference on Computer Vision, Vancouver, Canada, 2001: 416–423. doi: 10.1109/ICCV.2001.937655.
[24]	WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.1109/TIP.2003.819861.
[25]	LOSHCHILOV I and HUTTER F. Decoupled weight decay regularization[C]. 7th International Conference on Learning Representations, New Orleans, USA, 2019.