Unsupervised Underwater Image Enhancement Based on Feature Disentanglement

LIU Yancheng; DONG Zhangwei; ZHU Pengli; LIU Siyuan

doi:10.11999/JEIT211517

Volume 44 Issue 10

Oct. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(10): 3389-3398

LIU Yancheng, DONG Zhangwei, ZHU Pengli, LIU Siyuan. Unsupervised Underwater Image Enhancement Based on Feature Disentanglement[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3389-3398. doi: 10.11999/JEIT211517

Citation:

LIU Yancheng, DONG Zhangwei, ZHU Pengli, LIU Siyuan. Unsupervised Underwater Image Enhancement Based on Feature Disentanglement[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3389-3398. doi: 10.11999/JEIT211517

Citation:

PDF( 4882 KB)

Unsupervised Underwater Image Enhancement Based on Feature Disentanglement

doi: 10.11999/JEIT211517

1.
Marine Engineering College, Dalian Maritime University, Dalian 116026, China
2.
College of Marine Electrical Engineering, Dalian Maritime University, Dalian 116026, China

Funds: The National Natural Science Foundation of China (51979021, 51709028), The Natural Science Foundation of Liaoning Province (2019JH8, 10100045), The Fundamental Research Funds for the Central Universities (3132019317, 3132022218)

Received Date: 2021-12-15
Accepted Date: 2022-03-03
Rev Recd Date: 2022-02-27

Available Online: 2022-03-07

Publish Date: 2022-10-19

Abstract

Abstract

The absorption and scattering properties of the water medium cause different types of distortion in underwater images, which affects seriously the accuracy and effectiveness of subsequent processing. At present, underwater image enhancement methods with supervised learning rely on synthetic underwater paired image sets for training. However, the supervised learning methods are challenging to apply to practical application scenarios because the synthetic data may not accurately model the underlying physical mechanisms of underwater imaging. An unsupervised underwater image enhancement based on feature disentanglement is proposed. On the one hand, considering the difficulty and high cost of acquiring clear-unclear paired datasets in the same scene, a cycle generative adversarial network is employed to convert the underwater image enhancement problem into a style transfer problem to achieve unsupervised learning. On the other hand, the feature disentanglement method is combined to extract the style features and structure features separately to ensure the structural consistency of the images before and after enhancement. The experimental results show that the method can effectively recover the color and texture details of underwater images in the case of unpaired data training.
- Underwater image enhancement,
- Feature disentanglement,
- Generative adversarial network,
- Unsupervised learning

FullText(HTML)

References(36)

References

[1]	GHANI A S A and ISA N A M. Enhancement of low quality underwater image through integrated global and local contrast correction[J]. Applied Soft Computing, 2015, 37: 332–344. doi: 10.1016/j.asoc.2015.08.033
[2]	SINGH K, KAPOOR R, and SINHA S K. Enhancement of low exposure images via recursive histogram equalization algorithms[J]. Optik, 2015, 126(20): 2619–2625. doi: 10.1016/j.ijleo.2015.06.060
[3]	ZHANG Shu, WANG Ting, DONG Junyu, et al. Underwater image enhancement via extended multi-scale Retinex[J]. Neurocomputing, 2017, 245: 1–9. doi: 10.1016/j.neucom.2017.03.029
[4]	MERCADO M A, ISHII K, and AHN J. Deep-sea image enhancement using multi-scale retinex with reverse color loss for autonomous underwater vehicles[C]. OCEANS 2017-Anchorage, Anchorage, USA, 2017: 1–6.
[5]	GHANI A S A. Image contrast enhancement using an integration of recursive-overlapped contrast limited adaptive histogram specification and dual-image wavelet fusion for the high visibility of deep underwater image[J]. Ocean Engineering, 2018, 162: 224–238. doi: 10.1016/j.oceaneng.2018.05.027
[6]	LI Chongyi and GUO Jichang. Underwater image enhancement by dehazing and color correction[J]. Journal of Electronic Imaging, 2015, 24(3): 033023. doi: 10.1117/1.JEI.24.3.033023
[7]	JAFFE J S. Computer modeling and the design of optimal underwater imaging systems[J]. IEEE Journal of Oceanic Engineering, 1990, 15(2): 101–111. doi: 10.1109/48.50695
[8]	AKKAYNAK D and TREIBITZ T. A revised underwater image formation model[C]. The IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6723–6732.
[9]	ANWAR S and LI Chongyi. Diving deeper into underwater image enhancement: A survey[J]. Signal Processing:Image Communication, 2020, 89: 115978. doi: 10.1016/j.image.2020.115978
[10]	SUN Xin, LIU Lipeng, LI Qiong, et al. Deep pixel-to-pixel network for underwater image enhancement and restoration[J]. IET Image Processing, 2018, 13(3): 469–474. doi: 10.1049/iet-ipr.2018.5237
[11]	WANG Yang, ZHANG Jing, CAO Yang, et al. A deep CNN method for underwater image enhancement[C]. 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 2017: 1382–1386.
[12]	LI Chongyi, GUO Chunle, REN Wenqi, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2019, 29: 4376–4389. doi: 10.1109/TIP.2019.2955241
[13]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]. The 27th International Conference on Neural Information Processing Systems, Cambridge, UK, 2014: 2672–2680.
[14]	FABBRI C, ISLAM M J, and SATTAR J. Enhancing underwater imagery using generative adversarial networks[C]. 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 2018: 7159–7165.
[15]	GULRAJANI I, AHMED F, ARJOVSKY M, et al. Improved training of wasserstein GANs[C]. The 31st International Conference on Neural Information Processing Systems, Red Hook, USA, 2017: 5769–5779.
[16]	GUO Yecai, LI Hanyu, and ZHUANG Peixian. Underwater image enhancement using a multiscale dense generative adversarial network[J]. IEEE Journal of Oceanic Engineering, 2020, 45(3): 862–870. doi: 10.1109/JOE.2019.2911447
[17]	LI Jie, SKINNER K A, EUSTICE R M, et al. WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2017, 3(1): 387–394. doi: 10.1109/LRA.2017.2730363
[18]	LU Jingyu, LI Na, ZHANG Shaoyong, et al. Multi-scale adversarial network for underwater image restoration[J]. Optics & Laser Technology, 2019, 110: 105–113. doi: 10.1016/j.optlastec.2018.05.048
[19]	ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. The 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 2242–2251.
[20]	ISLAM M J, XIA Youya, and SATTAR J. Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3227–3234. doi: 10.1109/LRA.2020.2974710
[21]	MIRZA M and OSINDERO S. Conditional generative adversarial nets[J]. arXiv preprint arXiv: 1411.1784, 2014.
[22]	LEE H Y, TSENG H Y, HUANG Jiabin, et al. Diverse image-to-image translation via disentangled representations[C]. The 15th European Conference on Computer Vision (ECCV), Munich, Germany, 2018: 36–52.
[23]	HUANG Xun, LIU Mingyu, BELONGIE S, et al. Multimodal unsupervised image-to-image translation[C]. The 15th European Conference on Computer Vision (ECCV), Munich, Germany, 2018: 179–196.
[24]	LIU Siyuan, THUNG K H, QU Liangqiong, et al. Learning MRI artefact removal with unpaired data[J]. Nature Machine Intelligence, 2021, 3(1): 60–67. doi: 10.1038/s42256-020-00270-2
[25]	LOCATELLO F, BAUER S, LUCIC M, et al. Challenging common assumptions in the unsupervised learning of disentangled representations[C]. The 36th International Conference on Machine Learning, Long Beach, USA, 2019: 4114–4124.
[26]	LI Chuan and WAND M. Precomputed real-time texture synthesis with markovian generative adversarial networks[C]. 14th European Conference on Computer Vision, Amsterdam, Holland, 2016: 702–716.
[27]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. The 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778.
[28]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]. The 2017 IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, 2017: 936–944.
[29]	HUANG Xun and BELONGIE S. Arbitrary style transfer in real-time with adaptive instance normalization[C]. The 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 1510–1519.
[30]	KINGMA D P and WELLING M. Auto-encoding variational bayes[J]. arXiv preprint arXiv: 1312.6114v10, 2013.
[31]	KINGMA D P and BA J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv: 1412.6980v5, 2014.
[32]	PENG Y T and COSMAN P C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579–1594. doi: 10.1109/TIP.2017.2663846
[33]	SONG Wei, WANG Yan, HUANG Dongmei, et al. Enhancement of underwater images with statistical model of background light and optimization of transmission map[J]. IEEE Transactions on Broadcasting, 2020, 66(1): 153–169. doi: 10.1109/TBC.2019.2960942
[34]	LI Chongyi, GUO Jichang, and GUO Chunle. Emerging from water: Underwater image color correction based on weakly supervised color transfer[J]. IEEE Signal Processing Letters, 2018, 25(3): 323–327. doi: 10.1109/LSP.2018.2792050
[35]	YANG Miao and SOWMYA A. An underwater color image quality evaluation metric[J]. IEEE Transactions on Image Processing, 2015, 24(12): 6062–6071. doi: 10.1109/TIP.2015.2491020
[36]	PANETTA K, GAO Chen, and AGAIAN S. Human-visual-system-inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2016, 41(3): 541–551. doi: 10.1109/JOE.2015.2469915