Multi-scale Underwater Image Enhancement Network Based on Attention Mechanism

Ming FANG; Xiaohan LIU; Feiran FU

doi:10.11999/JEIT200836

Volume 43 Issue 12

Dec. 2021

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2021 > 43(12): 3513-3521

Ming FANG, Xiaohan LIU, Feiran FU. Multi-scale Underwater Image Enhancement Network Based on Attention Mechanism[J]. Journal of Electronics & Information Technology, 2021, 43(12): 3513-3521. doi: 10.11999/JEIT200836

Citation:

Ming FANG, Xiaohan LIU, Feiran FU. Multi-scale Underwater Image Enhancement Network Based on Attention Mechanism[J]. Journal of Electronics & Information Technology, 2021, 43(12): 3513-3521. doi: 10.11999/JEIT200836

Citation:

PDF( 5025 KB)

Multi-scale Underwater Image Enhancement Network Based on Attention Mechanism

doi: 10.11999/JEIT200836

Ming FANG^{2, 1},
Xiaohan LIU^{1
,
,},
Feiran FU¹

1.
School of Computer Science and Technology, Changchun University of Science and Technology, Changchun 130022, China
2.
School of Artificial Intelligence, Changchun University of Science and Technology, Changchun 130022, China

Funds: The Marine S&T Fund of Shandong Province for Pilot National Laboratory for Marine Science and Technology (Qingdao)(2018SDKJ0102-6)

Received Date: 2020-09-27
Rev Recd Date: 2021-04-25

Available Online: 2021-07-14

Publish Date: 2021-12-21

Abstract

Abstract

Due to the absorption and scattering, color degradation and detail blurring often occur in underwater images, which will affect the underwater visual tasks. A multi-scale underwater image enhancement network based on attention mechanism is designed in an end-to-end manner by synthesizing dataset closer to underwater images through underwater imaging model. In the network, pixel and channel attention mechanisms are introduced. A new multi-scale feature extraction module is designed to extract the features of different levels at the beginning of the network, and the output results are obtained via a convolution layer and an attention module with skip connections. Experimental results on multiple datasets show that the proposed method is effective in processing both synthetic and real underwater images. It can better recover the color and texture details of images compared with the existing methods.
- Underwater image enhancement,
- Deep learning,
- Attention mechanism,
- Multi-scale features

FullText(HTML)

References(34)

References

[1]	ZUIDERVELD K. Contrast Limited Adaptive Histogram Equalization[M]. Graphics Gems IV, Amsterdam: Elsevier, 1994: 474–485.
[2]	ANCUTI C, ANCUTI C O, HABER T, et al. Enhancing underwater images and videos by fusion[C]. 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, USA, 2012: 81–88.
[3]	HE Kaiming, SUN Jian, and TANG Xiaoou. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341–2353. doi: 10.1109/TPAMI.2010.168
[4]	LI Chongyi, GUO Jichang, CONG Runmin, et al. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior[J]. IEEE Transactions on Image Processing, 2016, 25(12): 5664–5677. doi: 10.1109/TIP.2016.2612882
[5]	杨爱萍, 郑佳, 王建, 等. 基于颜色失真去除与暗通道先验的水下图像复原[J]. 电子与信息学报, 2015, 37(11): 2541–2547. doi: 10.11999/JEIT150483 YANG Aiping, ZHENG Jia, WANG Jian, et al. Underwater image restoration based on color cast removal and dark channel prior[J]. Journal of Electronics &Information Technology, 2015, 37(11): 2541–2547. doi: 10.11999/JEIT150483
[6]	DREWS P L J, NASCIMENTO E R, BOTELHO S S C, et al. Underwater depth estimation and image restoration based on single images[J]. IEEE Computer Graphics and Applications, 2016, 36(2): 24–35. doi: 10.1109/MCG.2016.26
[7]	REN Wenqi, LIU Si, ZHANG Hua, et al. Single image dehazing via multi-scale convolutional neural networks[C]. The 14th European Conference on Computer Vision, Amsterdam, The Kingdom of the Netherlands, 2016: 154–169.
[8]	AKKAYNAK D and TREIBITZ T. A revised underwater image formation model[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6723–6732.
[9]	ANWAR S and LI Chongyi. Diving deeper into underwater image enhancement: A survey[EB/OL]. https://arxiv.org/abs/1907.07863, 2019.
[10]	LI Jie, SKINNER K A, EUSTICE R M, et al. WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2018, 3(1): 387–394. doi: 10.1109/LRA.2017.2730363
[11]	HASHISHO Y, ALBADAWI M, KRAUSE T, et al. Underwater color restoration using u-net denoising autoencoder[EB/OL]. https://arxiv.org/abs/1905.09000. 2019.
[12]	LI Chongyi, GUO Chunle, REN Wenqi, et al. An underwater image enhancement benchmark dataset and beyond[EB/OL]. https://arxiv.org/abs/1901.05495, 2019.
[13]	FABBRI C, ISLAM M J, and SATTAR J. Enhancing underwater imagery using generative adversarial networks[C]. 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 2018: 7159–7165.
[14]	ISLAM M J, XIA Y Y, and SATTAR J. Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3227–3234. doi: 10.1109/LRA.2020.2974710
[15]	ZHANG Tingting, LI Yujie, and TAKAHASHI S. Underwater image enhancement using improved generative adversarial network[J]. Concurrency and Computation Practice and Experience, 2020: e5841. doi: 10.1002/cpe.5841
[16]	PARK J, HAN D K, and KO H. Adaptive weighted multi-discriminator CycleGAN for underwater image enhancement[J]. Journal of Marine Science and Engineering, 2019, 7(7): 200. doi: 10.3390/jmse7070200
[17]	ANWAR S, LI Chongyi, and PORIKLI F. Deep underwater image enhancement[EB/OL]. https://arxiv.org/abs/1807.03528, 2018.
[18]	LI Chongyi, ANWAR S, and PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2020, 98: 107038. doi: 10.1016/j.patcog.2019.107038
[19]	SILBERMAN N, HOIEM D, KOHLI P, et al. Indoor segmentation and support inference from RGBD images[C]. The 12th European Conference on Computer Vision, Florence, Italy, 2012: 746–760.
[20]	ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 2242–2251.
[21]	CHIANG J Y and CHEN Y C. Underwater image enhancement by wavelength compensation and dehazing[J]. IEEE Transactions on Image Processing, 2012, 21(4): 1756–1769. doi: 10.1109/TIP.2011.2179666
[22]	DING Xueyan, WANG Yafei, YAN Yang, et al. Jointly adversarial network to wavelength compensation and dehazing of underwater images[EB/OL]. https://arxiv.org/abs/1907.05595, 2019.
[23]	ZHOU Yuan and YAN Kangming. Domain adaptive adversarial learning based on physics model feedback for underwater image enhancement[EB/OL]. https://arxiv.org/abs/2002.09315, 2020.
[24]	QIN Xu, WANG Zhilin, BAI Yuanchao, et al. FFA-Net: Feature fusion attention network for single image dehazing[EB/OL]. https://arxiv.org/abs/1911.07559, 2019.
[25]	柳长源, 王琪, 毕晓君, 等. 基于多通道多尺度卷积神经网络的单幅图像去雨方法[J]. 电子与信息学报, 2020, 42(9): 2285–2292. doi: 10.11999/JEIT190755 LIU Changyuan, WANG Qi, BI Xiaojun, et al. Research on rain removal method for single image based on multi-channel and multi-scale CNN[J]. Journal of Electronics &Information Technology, 2020, 42(9): 2285–2292. doi: 10.11999/JEIT190755
[26]	WANG Zhengyang and JI Shuiwang. Smoothed dilated convolutions for improved dense prediction[C]. The 24th ACM SIGKDD International Conference on Knowledge Discovery& Data Mining, London, UK, 2018: 2486–2495.
[27]	CHEN Dongdong, HE Mingming, FAN Qingnan, et al. Gated context aggregation network for image dehazing and deraining[C]. 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, USA, 2019: 1375–1383.
[28]	JOHNSON J, ALAHI A, and LI Feifei. Perceptual losses for real-time style transfer and super-resolution[C]. 14th European Conference on Computer Vision, Amsterdam, The Kingdom of the Netherlands, 2016: 694–711.
[29]	SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. International Conference on Learning Representations (ICLR), San Diego, USA, 2015: 1–5.
[30]	WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.1109/TIP.2003.819861
[31]	PANETTA K, GAO C, and AGAIAN S. Human-visual-system-inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2016, 41(3): 541–551. doi: 10.1109/JOE.2015.2469915
[32]	DREWS P JR, DO NASCIMENTO E, MORAES F, et al. Transmission estimation in underwater single images[C]. 2013 IEEE International Conference on Computer Vision Workshops, Sydney, Australia, 2013: 825–830.
[33]	PENG Y T and COSMAN P C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579–1594. doi: 10.1109/TIP.2017.2663846
[34]	SONG Wei, WANG Yan, HUANG Dongmei, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]. The 19th Pacific-Rim Conference on Multimedia, Hefei, China, 2018: 678–688.