基于多尺度级联网络的水下图像增强方法

米泽田; 晋洁; 李圆圆; 丁雪妍; 梁政; 付先平

doi:10.11999/JEIT220375

基于多尺度级联网络的水下图像增强方法

doi: 10.11999/JEIT220375

米泽田^1, ,,
晋洁¹,
李圆圆¹,
丁雪妍¹,
梁政³,
付先平^{1, 2}

1.
大连海事大学信息科学技术学院大连 116026
2.
鹏城实验室深圳 518000
3.
安徽大学互联网学院合肥 230039

基金项目: 国家自然科学基金(62176037)，辽宁省重点研发计划(201801728)

详细信息

作者简介:
米泽田：女，副教授，研究方向为图像处理和机器学习等

晋洁：女，硕士生，研究方向为图像处理等

李圆圆：女，硕士生，研究方向为图像处理等

丁雪妍：女，博士生，研究方向为水下图像增强等

梁政：男，讲师，研究方向为计算机视觉和机器学习等

付先平：男，教授，研究方向为图像处理和机器学习等

通讯作者:
米泽田　mizetian_group@163.com

中图分类号: TN911.73
计量
- 文章访问数: 767
- HTML全文浏览量: 200
- PDF下载量: 203
- 被引次数: 0
出版历程
- 收稿日期: 2022-04-01
- 修回日期: 2022-06-23
- 网络出版日期: 2022-06-29
- 刊出日期: 2022-10-19

Underwater Image Enhancement Method Based on Multi-scale Cascade Network

1.
College of Information Science and Technology, Dalian Maritime University, Dalian 116026, China
2.
Peng Cheng Laboratory, Shenzhen 518000, China
3.
College of Internet, Anhui University, Hefei 230039, China

Funds: The National Natural Science Foundation of China (62176037), The Foundation of Liaoning Province Key Research and Development Program (201801728)

摘要

摘要: 针对水下图像由于光吸收、后向散射等因素导致的严重色偏、细节丢失等问题，该文提出一种基于多尺度级联网络的水下图像增强方法。针对单一网络特征利用不全面导致的图像梯度消失问题，该方法通过级联多尺度原始图像与相应的特征图像，以获得更优异的细节保持效果，并实现从较浅层到较深层快速预测残差的能力。此外，引入联合密集网络块和递归块，通过特征重用有效解决多尺度网络参数过多的问题。为有效解决单一损失造成的图像细节恢复不均的问题，提出Charbonnier和结构相似度 (SSIM) 联合损失函数。经仿真实验分析，所提网络在处理水下图像严重色偏、细节丢失等方面都取得了显著的效果。
- 水下图像增强 /
- 多尺度级联网络 /
- 多尺度特征提取 /
- 梯度消失
Abstract: Focusing on the serious color shift and loss of details caused by light absorption, backscattering and other factors in underwater images, an underwater image enhancement method based on multi-scale cascaded network is proposed in this paper. For the image gradient dissipation caused by incomplete utilization of features via single network, better details are preserved by cascading multi-scale original images and corresponding feature images, and rapid prediction of residuals from shallower layers to deeper layers can be realized at the same time. In addition, joint dense network block and recursive block are introduced to avoid effectively the problem of excessive parameters introduced by conventional multi-scale network through feature reuse. A joint loss function of Charbonnier and the Structural SIMilarity (SSIM) is proposed to solve effectively the problem of uneven restoration of image details caused by a single loss. The simulation experiments show that the proposed network has achieved excellent results in dealing with severe color shift and loss of details.
- Underwater image enhancement /
- Multi-scale cascade network /
- Multi-scale feature extraction /
- Gradient disappears

HTML全文

图 1 多尺度级联网络结构

下载: 全尺寸图片幻灯片

图 2 卷积块结构

下载: 全尺寸图片幻灯片

图 3 Recursive block结构

下载: 全尺寸图片幻灯片

图 4 UIEB数据集上不同方法的比较

下载: 全尺寸图片幻灯片

图 5 EUVP1数据集上不同方法的比较

下载: 全尺寸图片幻灯片

图 6 EUVP2数据集上不同方法的比较

下载: 全尺寸图片幻灯片

图 7 EUVP3数据集上不同方法的比较

下载: 全尺寸图片幻灯片

图 8 在UIEB数据集上其他情况的比较

下载: 全尺寸图片幻灯片

表 1 4种不同水下数据集的划分

	UIEB	EUVP1	EUVP2	EUVP3
训练图像对	790	4995	3330	1966
测试图像对	100	555	370	219
总图像对	890	5550	3700	2185

下载: 导出CSV

表 2 对网络框架结构的定量评价

	PSNR	SSIM	Blur	PCQI
A + C	21.1980	0.6925	0.2742	0.9553
A + B + C	21.1722	0.7017	0.2702	0.9385

下载: 导出CSV

表 3 对多尺度子网络层数的定量评价

	PSNR	SSIM	Blur	PCQI
2层	20.3955	0.6789	0.2653	0.9370
3层(本文)	21.1722	0.7017	0.2702	0.9385
4层	20.6914	0.6981	0.2777	0.9628
5层	19.6474	0.6290	0.2741	0.9220

下载: 导出CSV

表 4 对多尺度子网络中递归块的定量评价

块数	PSNR	SSIM	Blur	PCQI
2	20.3971	0.6776	0.2733	0.9551
3(本文)	21.1722	0.7017	0.2702	0.9385
4	21.4235	0.7000	0.2799	0.9435

下载: 导出CSV

表 5 对损失函数的定量评价

	PSNR	SSIM	Blur	PCQI
l₁	7.1387	0.0028	NaN	0.1345
Charbonnier	21.2840	0.6687	0.2707	0.9194
l₁ + SSIM	7.1387	0.0028	NaN	0.1345
Charbonnier + SSIM(本文)	21.1722	0.7017	0.2702	0.9385

下载: 导出CSV

表 6 在4种数据集上的PSNR定量评价(dB)

	UIEB	EUVP1	EUVP2	EUVP3
MIP	17.7824	23.2208	19.0353	19.6164
SMBLOT	20.4385	19.9150	20.3344	19.2797
UDCP	12.5706	16.6085	18.6297	16.8542
ULAP	20.8729	23.8800	22.4006	22.8187
RD	20.9269	20.2605	19.2846	18.1743
IBLA	17.0296	23.7864	21.7833	22.7280
UWCNN	15.3285	23.2881	18.6631	20.0642
UWGAN	21.6163	13.3417	13.0353	10.5894
Pix2pix	23.0141	30.1325	25.1247	28.1732
CycleGAN	8.6081	22.0597	8.6905	25.1744
本文	21.1722	28.5354	23.3623	22.9755

下载: 导出CSV

表 7 在4种数据集上的SSIM定量评价

	UIEB	EUVP1	EUVP2	EUVP3
MIP	0.4868	0.4155	0.4879	0.5424
SMBLOT	0.6571	0.4999	0.5637	0.6393
UDCP	0.4753	0.3282	0.4597	0.5401
ULAP	0.5373	0.4940	0.6054	0.7138
RD	0.6428	0.5116	0.5375	0.5079
IBLA	0.5188	0.4463	0.4585	0.6662
UWCNN	0.4660	0.5747	0.6421	0.6065
UWGAN	0.7428	0.0384	0.1218	-0.0232
Pix2pix	0.6512	0.3390	0.3858	0.6748
CycleGAN	0.1084	0.4879	0.0318	0.7761
本文	0.7017	0.6976	0.7410	0.7307

下载: 导出CSV

表 8 在4种数据集上的PCQI定量评价

	UIEB	EUVP1	EUVP2	EUVP3
MIP	0.6981	0.9358	0.6637	0.6033
SMBLOT	0.7830	1.0196	0.7697	0.6867
UDCP	0.5646	0.8061	0.5988	0.5058
ULAP	0.7359	0.9941	0.7159	0.6484
RD	0.8435	1.0176	0.7684	0.6724
IBLA	0.6453	0.9681	0.6911	0.6212
UWCNN	0.4277	0.7683	0.4445	0.4423
UWGAN	0.7686	0.0448	0.0499	0.1272
Pix2pix	0.5379	0.8915	0.5498	0.5996
CycleGAN	0.2957	0.5093	0.3557	0.5901
本文	0.9385	0.9209	0.7483	0.7753

下载: 导出CSV

表 9 在4种数据集上的Blur定量评价

	UIEB	EUVP1	EUVP2	EUVP3
MIP	0.2840	0.2513	0.3228	0.3737
SMBLOT	0.2881	0.2397	0.3347	0.3778
UDCP	0.2844	0.2405	0.3424	0.3814
ULAP	0.2898	0.2518	0.3398	0.3858
RD	0.2746	0.2427	0.3252	0.3577
IBLA	0.2870	0.2525	0.3403	0.3884
UWCNN	0.2956	0.2518	0.3428	0.3876
UWGAN	0.3315	0.1664	0.1794	0.2880
Pix2pix	0.3319	0.2613	0.3769	0.4057
CycleGAN	0.3148	0.3086	0.3548	0.3305
本文	0.2702	0.2682	0.3362	0.2784

下载: 导出CSV

表 10 不同卷积神经网络模型参数量分析

	DenseNet121	CycleGAN	UWCNN	Pix2pix	UWGAN	MFFN	UMUEN	本文
参数量(M)	7.98	107.95	0.15	54.41	1.93	11.97	29.2	1.42

下载: 导出CSV

参考文献(29)

[1]	PIZER S M, AMBURN E P, AUSTIN J D, et al. Adaptive histogram equalization and its variations[J]. Computer Vision, Graphics, and Image Processing, 1987, 39(3): 355–368. doi: 10.1016/s0734-189x(87)80186-x
[2]	PIZER S M, JOHNSTON R E, ERICKSEN J P, et al. Contrast-limited adaptive histogram equalization: Speed and effectiveness[C]. The First Conference on Visualization in Biomedical Computing, Atlanta, USA, 1990: 337–345.
[3]	RIZZI A, GATTA C, and MARINI D. Color correction between gray world and white patch[J]. SPIE, 2002, 4662: 1–10.
[4]	VAN DE WEIJER J, GEVERS T, and GIJSENIJ A. Edge-based color constancy[J]. IEEE Transactions on Image Processing, 2007, 16(9): 2207–2214. doi: 10.1109/TIP.2007.901808
[5]	SINGH G, JAGGI N, VASAMSETTI S, et al. Underwater image/video enhancement using wavelet based color correction (WBCC) method[C]. 2015 IEEE Underwater Technology, Chennai, India, 2015: 1–5.
[6]	MUNIRAJ M and DHANDAPANI V. Underwater image enhancement by combining color constancy and dehazing based on depth estimation[J]. Neurocomputing, 2021, 460: 211–230. doi: 10.1016/j.neucom.2021.07.003
[7]	ZHUANG Peixian, LI Chongyi, and WU Jiamin. Bayesian retinex underwater image enhancement[J]. Engineering Applications of Artificial Intelligence, 2021, 101: 104171. doi: 10.1016/j.engappai.2021.104171
[8]	GALDRAN A, PARDO D, PICÓN A, et al. Automatic Red-Channel underwater image restoration[J]. Journal of Visual Communication and Image Representation, 2015, 26: 132–145. doi: 10.1016/j.jvcir.2014.11.006
[9]	LI Chongyi, GUO Jichang, CONG Runmin, et al. Underwater image enhancement by Dehazing with minimum information loss and histogram distribution prior[J]. IEEE Transactions on Image Processing, 2016, 25(12): 5664–5677. doi: 10.1109/TIP.2016.2612882
[10]	LI Chongyi, ANWAR S, and PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2020, 98: 107038. doi: 10.1016/j.patcog.2019.107038
[11]	WANG Nan, ZHOU Yabin, HAN Fenglei, et al. UWGAN: Underwater GAN for real-world underwater color restoration and Dehazing[J/OL]. https://arxiv.org/abs/1912.10269, 2019.
[12]	ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017: 2242–2251.
[13]	GUO Yecai, LI Hanyu, and ZHUANG Peixian. Underwater image enhancement using a multiscale dense generative adversarial network[J]. IEEE Journal of Oceanic Engineering, 2020, 45(3): 862–870. doi: 10.1109/JOE.2019.2911447
[14]	HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]. The IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 2261–2269.
[15]	LAI Weisong, HUANG Jiabin, AHUJA N, et al. Fast and accurate image super-resolution with deep Laplacian pyramid networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(11): 2599–2613. doi: 10.1109/TPAMI.2018.2865304
[16]	WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.1109/TIP.2003.819861
[17]	LI Chongyi, GUO Chunle, REN Wenqi, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020, 29: 4376–4389. doi: 10.1109/TIP.2019.2955241
[18]	ISLAM M J, XIA Youya, and SATTAR J. Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3227–3234. doi: 10.1109/LRA.2020.2974710
[19]	WANG Shiqi, MA Kede, YEGANEH H, et al. A patch-structure representation method for quality assessment of contrast changed images[J]. IEEE Signal Processing Letters, 2015, 22(12): 2387–2390. doi: 10.1109/LSP.2015.2487369
[20]	CRETE F, DOLMIERE T, LADRET P, et al. The blur effect: Perception and estimation with a new no-reference perceptual blur metric[J]. SPIE, 2007, 6492, 64920I.
[21]	CARLEVARIS-BIANCO N, MOHAN A, and EUSTICE R M. Initial results in underwater single image dehazing[C]. OCEANS 2010 MTS/IEEE Seattle, Seattle, USA, 2010: 1–8.
[22]	SONG Wei, WANG Yan, HUANG Dongmei, et al. Enhancement of underwater images with statistical model of background light and optimization of transmission map[J]. IEEE Transactions on Broadcasting, 2020, 66(1): 153–169. doi: 10.1109/TBC.2019.2960942
[23]	DREWS P, DO NASCIMENTO E, MORAES F, et al. Transmission estimation in underwater single images[C]. 2013 IEEE International Conference on Computer Vision Workshops, Sydney, Australia, 2013: 825–830.
[24]	SONG Wei, WANG Yan, HUANG Dongmei, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]. 19th Pacific-Rim Conference on Advances in Multimedia Information Processing, Hefei, China, 2018: 678–688.
[25]	GHANI A S A and ISA N A. Underwater image quality enhancement through composition of dual-intensity images and Rayleigh-stretching[J]. SpringerPlus, 2014, 3: 757. doi: 10.1186/2193-1801-3-757
[26]	PENG Y T and COSMAN P C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579–1594. doi: 10.1109/TIP.2017.2663846
[27]	ISOLA P, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks[C]. The IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 5967–5976.
[28]	CHEN Renzhang, CAI Zhanchuan, and CAO Wei. MFFN: An underwater sensing scene image enhancement method based on multiscale feature fusion network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 4205612. doi: 10.1109/TGRS.2021.3134762
[29]	DING Xuan, DONG Hang, GUO Yu, et al. Uncertainly guided multi-scale underwater image enhancement network with tone mapping block[C]. 2021 China Automation Congress (CAC), Beijing, China, 2021: 7172–7176.