Low-light Image Enhancement Method Based on Shifted Window Multi-head Self-attention U-shaped Network

SUN Bangyong; ZHAO Xingyun; WU Siyuan; YU Tao

doi:10.11999/JEIT211131

Volume 44 Issue 10

Oct. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(10): 3399-3408

SUN Bangyong, ZHAO Xingyun, WU Siyuan, YU Tao. Low-light Image Enhancement Method Based on Shifted Window Multi-head Self-attention U-shaped Network[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3399-3408. doi: 10.11999/JEIT211131

Citation:

SUN Bangyong, ZHAO Xingyun, WU Siyuan, YU Tao. Low-light Image Enhancement Method Based on Shifted Window Multi-head Self-attention U-shaped Network[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3399-3408. doi: 10.11999/JEIT211131

Citation:

PDF( 9387 KB)

Low-light Image Enhancement Method Based on Shifted Window Multi-head Self-attention U-shaped Network

doi: 10.11999/JEIT211131

SUN Bangyong^{1, 2},
ZHAO Xingyun¹,
WU Siyuan²,
YU Tao^{2
,
,}

1.
College of Printing, Packaging and Digital Media, Xi’an University of Technology, Xi’an 710048, China
2.
Key Laboratory of Spectral Imaging Technology, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China

Funds: The National Natural Science Foundation of China (62076199), The Key R&D Project of Shaan'xi Province (2021GY-027), The Key Laboratory of Spectral Imaging Technology of Chinese Academy of Sciences (LSIT201801D)

Received Date: 2021-10-14
Accepted Date: 2021-12-07
Rev Recd Date: 2021-12-03

Available Online: 2021-12-10

Publish Date: 2022-10-19

Abstract

Abstract

Considering the difficult problems of brightness enhancement, noise suppression and maintaining texture color consistency in the low-light image enhancement model, a low-light image enhancement method based on the shifted window self-attention mechanism is proposed. Based on the U-shaped structure and the multi-head self-attention model of shifted windows, an image enhancement network composed of encoders, decoders and jump connections is constructed. The feature extraction advantages of the self-attention mechanism are applied to the field of low-light image enhancement and long-term dependence between image feature information is established, which can obtain global features effectively. The proposed method is compared width current popular algorithms in quantitative and qualitative comparison experiments, subjectively, the brightness of the image and noise suppression are significantly improved, and simultaneously better maintains the color information that highlights the texture details by the proposed method. In terms of objective indicators such as Peak Signal-to-Noise Ratio(PSNR), Structural SIMilarity index(SSIM), and Learned Perceptual Image Patch Similarity (LPIPS), which are improved 0.35 dB, 0.041 and 0.031 respectively compared with the optimal values of other methods. The experimental results show that the subjective perception quality and objective evaluation indicators of low-light images can be effectively improved by the proposed method, indicating a certain application value
- Image processing,
- Deep learning,
- Low-light image enhancement

FullText(HTML)

References(23)

References

[1]	KIM W, JEONG J, and YOU J. Contrast enhancement using histogram equalization based on logarithmic mapping[J]. Optical Engineering, 2012, 51(6): 067002. doi: 10.1117/1.OE.51.6.067002
[2]	CELIK T. Spatial entropy-based global and local image contrast enhancement[J]. IEEE Transactions on Image Processing, 2014, 23(12): 5298–5308. doi: 10.1109/TIP.2014.2364537
[3]	JOBSON D J, RAHMAN Z, and WOODELL G A. Properties and performance of a center/surround retinex[J]. IEEE Transactions on Image Processing, 1997, 6(3): 451–462. doi: 10.1109/83.557356
[4]	JOBSON D J, RAHMAN Z, and WOODELL G A. A multiscale retinex for bridging the gap between color images and the human observation of scenes[J]. IEEE Transactions on Image Processing, 1997, 6(7): 965–976. doi: 10.1109/83.597272
[5]	WANG Shuhang, ZHENG Jin, HU Haimiao, et al. Naturalness preserved enhancement algorithm for non-uniform illumination images[J]. IEEE Transactions on Image Processing, 2013, 22(9): 3538–3548. doi: 10.1109/TIP.2013.2261309
[6]	LI Mading, LIU Jiaying, YANG Wenhan, et al. Structure-revealing low-light image enhancement via robust retinex model[J]. IEEE Transactions on Image Processing, 2018, 27(6): 2828–2841. doi: 10.1109/TIP.2018.2810539
[7]	LORE K G, AKINTAYO A, and SARKAR S. LLNet: A deep autoencoder approach to natural low-light image enhancement[J]. Pattern Recognition, 2017, 61: 650–662. doi: 10.1016/j.patcog.2016.06.008
[8]	WEI Chen, WANG Wenjing, YANG Wenhan, et al. Deep retinex decomposition for low-light enhancement[C]. 2018 British Machine Vision Conference, Newcastle, UK, 2018: 1–12.
[9]	ZHANG Yonghua, ZHANG Jiawan, and GUO Xiaojie. Kindling the darkness: A practical low-light image enhancer[C]. The 27th ACM International Conference on Multimedia, Nice, France, 2019: 1632–1640.
[10]	CHEN Chen, CHEN Qifeng, XU Jia, et al. Learning to see in the dark[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 3291–3300.
[11]	WANG Ruixing, ZHANG Qing, FU C, et al. Underexposed photo enhancement using deep illumination estimation[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 6842–6850.
[12]	JIANG Yifan, GONG Xinyu, LIU Ding, et al. EnlightenGAN: Deep light enhancement without paired supervision[J]. IEEE Transactions on Image Processing, 2021, 30: 2340–2349. doi: 10.1109/TIP.2021.3051462
[13]	RONNEBERGER O, FISCHER P, and BROX T. U-Net: Convolutional networks for biomedical image segmentation[C]. The 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Munich, Germany, 2015: 234–241.
[14]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. The 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 6000–6010.
[15]	LAI Weisheng, HUANG Jiabin, AHUJA N, et al. Fast and accurate image super-resolution with deep laplacian pyramid networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(11): 2599–2613. doi: 10.1109/TPAMI.2018.2865304
[16]	WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.1109/TIP.2003.819861
[17]	JOHNSON J, ALAHI A, and LI F F. Perceptual losses for real-time style transfer and super-resolution[C]. The 14th European Conference on Computer Vision, Amsterdam, The Netherlands, 2016: 694–711.
[18]	LUO M R, CUI G, and RIGG B. The development of the CIE 2000 colour-difference formula: CIEDE2000[J]. Color Research and Application, 2001, 26(5): 340–350. doi: 10.1002/col.1049
[19]	ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 586–595.
[20]	LV Feifan, LU Feng, WU Jianhua, et al. MBLLEN: Low-light image/video enhancement using CNNs[C]. 2018 British Machine Vision Conference, Newcastle, UK, 2018: 1–13.
[21]	WANG Wenjing, WEI Chen, YANG Wenhan, et al. GLADNet: Low-light enhancement network with global awareness[C]. The 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi'an, China, 2018: 751–755.
[22]	ZHANG Yu, DI Xiaoguang, ZHANG Bin, et al. Self-supervised image enhancement network: Training with low light images only[EB/OL]. https://arxiv.org/pdf/2002.11300.pdf, 2020.
[23]	GUO Chunle, LI Chongyi, GUO Jichang, et al. Zero-reference deep curve estimation for low-light image enhancement[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 1777–1786.