基于雾线先验的时空关联约束视频去雾算法

姚婷婷; 梁越; 柳晓鸣; 胡青

doi:10.11999/JEIT190403

基于雾线先验的时空关联约束视频去雾算法

doi: 10.11999/JEIT190403

大连海事大学信息科学技术学院大连 116026

基金项目: 中央高校基本科研业务费专项资金(3132020208)，国家自然科学基金(31700742)

详细信息

作者简介:
姚婷婷：女，1988年生，讲师，研究方向为计算机视觉与图像处理等

梁越：男，1996年生，硕士生，研究方向为雾天视频处理

柳晓鸣：男，1959年生，教授，研究方向为海上交通电子信息处理、雷达信号处理等

胡青：男，1978年生，教授，研究方向为海事信息传输、自动识别系统等

通讯作者:
姚婷婷　ytt1030@dlmu.edu.cn

¹⁾ SfM算法程序可以从网址: http://ccwu.me/vsfm/获得
中图分类号: TN911.73, TP391
计量
- 文章访问数: 1916
- HTML全文浏览量: 815
- PDF下载量: 113
- 被引次数: 0
出版历程
- 收稿日期: 2019-06-05
- 修回日期: 2020-06-22
- 网络出版日期: 2020-07-17
- 刊出日期: 2020-11-16

Video Dehazing Algorithm via Haze-line Prior with Spatiotemporal Correlation Constraint

College of Information Science and Technology, Dalian Maritime University, Dalian 116026, China

Funds: The Fundamental Research Funds for the Central Universities (3132020208), The National Natural Science Foundation of China (31700742)

摘要

摘要: 现有视频去雾算法由于缺少对视频结构关联约束和帧间一致性分析，容易导致连续帧去雾结果在颜色和亮度上存在突变，同时去雾后的前景目标边缘区域也容易出现退化现象。针对上述问题，该文提出一种基于雾线先验的时空关联约束视频去雾算法，通过引入每帧图像在空间邻域中具有的结构关联性和时间邻域中具有的连续一致性，提高视频去雾算法的求解准确性和鲁棒性。算法首先使用暗通道先验估计每帧图像的大气光向量，并结合雾线先验求取初始透射率图。然后引入加权最小二乘边缘保持平滑滤波器对初始透射率图进行空间平滑，消除奇异点和噪声对估计结果的影响。进一步利用相机参数刻画连续帧间透射率图的时序变化规律，对独立求取的每帧透射率图进行时序关联修正。最后根据雾图模型获得最终的视频去雾结果。定性和定量的对比实验结果表明，该算法下视频去雾结果的帧间过渡更加自然，同时对每一帧图像的色彩还原更加准确，图像边缘的细节信息显示也更加丰富。
- 视频去雾 /
- 空间平滑 /
- 时序关联约束 /
- 雾线先验
Abstract: Because of the existent video dehazing algorithm lacks the analysis of the video structure correlation constraint and inter-frame consistency, it is easy to cause the dehazing results of continuous frames to have sudden changes in color and brightness. Meanwhile, the edge of foreground target is also prone to degradation. Focus on the aforementioned problems, a novel video dehazing algorithm via haze-line prior with spatiotemporal correlation constraint is proposed, which improves the accuracy and robustness of video dehazing result by bringing the structural relevance and temporal consistency of each frame. Firstly, the dark channel and haze-line prior are utilized to estimate the atmospheric light vector and initial transmission image of each frame. Then a weighted least square edge preserving smoothing filter is introduced to smooth the initial transmission image and eliminate the influence of singularities and noises on the estimated results. Furthermore, the camera parameters are calculated to describe the time series variation of the transmission image between continuous frames, and the independently obtained transmission image of each frame is corrected with temporal correlation constraint. Finally, according to the physical model, the video dehazing results are obtained. The experimental results of qualitative and quantitative comparison show that the proposed algorithm could make the inter-frame transition more smooth, and restore the color of each frame more accurately. Besides, more details are displayed at the edge of the dehazing results.
- Video dehazing /
- Spatial smoothing /
- Temporal correlation constraint /
- Haze-line prior
¹⁾ SfM算法程序可以从网址: http://ccwu.me/vsfm/获得

HTML全文

图 1 雾线示意图

下载: 全尺寸图片幻灯片

图 2 本文算法总体框图

下载: 全尺寸图片幻灯片

图 3 同一帧各阶段透射率图对比

下载: 全尺寸图片幻灯片

图 4 视频Ship和Beach连续帧下去雾结果对比

下载: 全尺寸图片幻灯片

图 5 单帧图像去雾结果对比

下载: 全尺寸图片幻灯片

表 1 各算法在不同评价指标下的性能对比

视频集	算法	VCM	SSIM	HCC	信息熵	UQI
Bali	文献[16]算法	47.7398±4.3502	0.6614±0.0459	–0.3173±0.0485	6.8571±0.1641	0.5923±0.0425
	文献[17]算法	30.1254±5.6277	0.5526±0.0215	–0.2960±0.0326	7.3794±0.0792	0.4669±0.0225
	文献[19]算法	41.6247±4.9448	0.8693±0.0041	–0.0813±0.0581	7.5121±0.0447	0.8061±0.0086
	文献[20]算法	37.4402±4.0238	0.6221±0.0214	–0.1968±0.1061	6.5753±0.1445	0.5889±0.0335
	文献[21]算法	49.7812±4.4782	0.7001±0.1116	–0.0312±0.0639	7.5413±0.0572	0.8819±0.0101
	本文算法	51.3852±6.3223	0.6679±0.0249	–0.2532±0.0251	7.9253±0.1322	0.8938±0.0285
Blenheim	文献[16]算法	35.5897±2.2001	0.8686±0.0129	0.0667±0.0165	6.5025±0.1967	0.7960±0.0258
	文献[17]算法	37.5815±1.6224	0.8260±0.0366	0.4688±0.0945	7.0697±0.0903	0.7661±0.0373
	文献[19]算法	26.0153±1.9259	0.9123±0.0035	0.4406±0.0231	7.0390±0.0716	0.8332±0.0209
	文献[20]算法	18.2786±2.2769	0.8215±0.0262	0.2759±0.0339	6.4305±0.0602	0.7721±0.0449
	文献[21]算法	64.9899±1.7827	0.6632±0.0094	0.0377±0.0135	6.2305±0.1637	0.7239±0.0108
	本文算法	40.0056±0.9116	0.9764±0.0022	0.7940±0.0969	7.4379±0.0434	0.9549±0.0131
Playground	文献[16]算法	50.2886±6.4619	0.8954±0.0129	–0.0276±0.0249	5.9694±0.4580	0.9021±0.0123
	文献[17]算法	31.9030±8.1211	0.7788±0.0575	0.0115±0.1177	7.4964±0.0911	0.7529±0.0703
	文献[19]算法	43.6243±3.5659	0.9205±0.0086	0.1664±0.0519	7.2406±0.1060	0.8967±0.0142
	文献[20]算法	35.5237±3.2426	0.7807±0.0182	–0.1450±0.0595	6.9652±0.0930	0.7728±0.0309
	文献[21]算法	51.8038±5.3890	0.6917±0.0258	–0.0457±0.0417	6.8567±0.2089	0.8213±0.0204
	本文算法	54.4761±10.9746	0.9546±0.0230	0.0587±0.0506	7.1156±0.1721	0.9379±0.0421
Stele	文献[16]算法	30.7638±15.7369	0.3649±0.0288	0.4263±0.1234	6.8163±0.0828	0.7349±0.0476
	文献[17]算法	44.2502±3.1428	0.7392±0.0199	0.0577±0.0443	7.1132±0.0828	0.7665±0.0296
	文献[19]算法	51.0145±5.4531	0.8529±0.0109	0.3319±0.0846	6.5512±0.1477	0.7671±0.0174
	文献[20]算法	40.8246±4.8187	0.8214±0.0157	0.2711±0.1034	6.2814±0.1242	0.7805±0.0233
	文献[21]算法	75.7723±3.8898	0.5875±0.0166	0.0227±0.0419	7.3425±0.1065	0.7881±0.0252
	本文算法	29.9452±3.0779	0.9045±0.0222	0.4791±0.1163	7.9275±0.0933	0.8580±0.0361
Motocycle	文献[16]算法	35.9520±15.9557	0.6854±0.0389	–0.0101±0.0254	6.9864±0.0507	0.5594±0.0292
	文献[17]算法	49.8669±5.5066	0.7500±0.0324	0.3960±0.0882	7.4345±0.0583	0.8401±0.0319
	文献[19]算法	33.4659±11.3533	0.8693±0.0047	0.2797±0.0475	7.0304±0.0646	0.8535±0.0114
	文献[20]算法	25.9303±7.0241	0.3705±0.0650	–0.3172±0.3156	5.7929±0.3812	0.1670±0.0547
	文献[21]算法	69.1006±4.9147	0.5231±0.0146	0.0111±0.0299	7.6674±0.1062	0.7442±0.0219
	本文算法	52.0758±8.0932	0.7720±0.0281	0.5653±0.0896	7.2598±0.0992	0.8955±0.0261
Ship	文献[16]算法	38.8112±11.4819	0.7648±0.1141	0.0505±0.1817	7.5924±0.1369	0.6952±0.1243
	文献[17]算法	33.9529±3.1727	0.8063±0.0162	0.0147±0.0551	7.5878±0.0791	0.7969±0.0267
	文献[19]算法	35.1544±10.5172	0.8396±0.0104	0.0313±0.0449	7.5395±0.0431	0.7682±0.0134
	文献[20]算法	33.8128±3.7508	0.5602±0.0496	–0.3243±0.0212	6.8355±0.1586	0.3912±0.0669
	文献[21]算法	53.0535±4.7626	0.6343±0.0167	–0.1062±0.0099	6.8996±0.1023	0.7783±0.0097
	本文算法	46.0955±3.6169	0.8365±0.0125	0.2137±0.0863	7.8230±0.0131	0.7985±0.0206
Beach	文献[16]算法	16.2818±4.7095	0.9786±0.0096	0.7025±0.0864	7.3837±0.0457	0.9702±0.0133
	文献[17]算法	38.2337±5.4335	0.7396±0.0096	–0.2957±0.0371	7.4769±0.0844	0.6582±0.0159
	文献[19]算法	9.4401±2.4871	0.8823±0.0050	0.0043±0.0454	7.4627±0.0237	0.8078±0.0062
	文献[20]算法	22.7816±3.3244	0.4762±0.4821	–0.3919±0.0207	6.4422±0.2008	0.2868±0.0615
	文献[21]算法	32.7816±8.8543	0.7221±0.0207	0.0181±0.0082	7.4134±0.0851	0.8845±0.0031
	本文算法	28.2158±5.8622	0.9802±0.0154	0.8117±0.0881	7.8108±0.0240	0.9415±0.0197

下载: 导出CSV

表 2 各算法计算效率对比

算法	文献[16]算法	文献[17]算法	文献[19]算法	文献[20]算法	文献[21]算法	本文算法
时间(s)	182.2654	1.0985	0.1076	1.8250	0.2601	1.0502

下载: 导出CSV

参考文献(21)

XU Yong, WEN Jie, FEI Lunke, et al. Review of video and image defogging algorithms and related studies on image restoration and enhancement[J]. IEEE Access, 2015, 4: 165–188. doi: 10.1109/ACCESS.2015.2511558

YU Tianhe, MENG Xue, ZHU Ming, et al. An improved multi-scale retinex fog and haze image enhancement method[C]. 2016 International Conference on Information System and Artificial Intelligence, Hong Kong, China, 2016: 557–560. doi: 10.1109/ISAI.2016.0124.

LI Yaning, WANG Junping, GAO Kang, et al. Fast morphological filtering haze removal method from a single image[J]. Journal of Computational Information Systems, 2015, 11(16): 5799–5806. doi: 10.12733/jcis14861

QIAO Tong, REN Jinchang, WANG Zheng, et al. Effective denoising and classification of hyperspectral images using curvelet transform and singular spectrum analysis[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(1): 119–133. doi: 10.1109/TGRS.2016.2598065

黄果, 许黎, 陈庆利, 等. 非局部多尺度分数阶微分图像增强算法研究[J]. 电子与信息学报, 2019, 41(12): 2972–2979. doi: 10.11999/JEIT190032

HUANG Guo, XU Li, CHEN Qingli, et al. Research on non-local multi-scale fractional differential image enhancement algorithm[J]. Journal of Electronics &Information Technology, 2019, 41(12): 2972–2979. doi: 10.11999/JEIT190032

HE Kaiming, SUN Jian, and TANG Xiaoou. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341–2353. doi: 10.1109/TPAMI.2010.168

杨爱萍, 王南, 庞彦伟, 等. 人工光源条件下夜间雾天图像建模及去雾[J]. 电子与信息学报, 2018, 40(6): 1330–1337. doi: 10.11999/JEIT170704

YANG Aiping, WANG Nan, PANG Yanwei, et al. Nighttime haze removal based on new imaging model with artificial light sources[J]. Journal of Electronics &Information Technology, 2018, 40(6): 1330–1337. doi: 10.11999/JEIT170704

江巨浪, 孙伟, 王振东, 等. 基于透射率权值因子的雾天图像融合增强算法[J]. 电子与信息学报, 2018, 40(10): 2388–2394. doi: 10.11999/JEIT171032

JIANG Julang, SUN Wei, WANG Zhendong, et al. Integrated enhancement algorithm for hazy image using transmittance as weighting factor[J]. Journal of Electronics &Information Technology, 2018, 40(10): 2388–2394. doi: 10.11999/JEIT171032

MA Xiao, SHAO Limin, XU Guanlei, et al. Intelligent defogging method based on clustering and dark channel prior[C]. 2018 IEEE 3rd International Conference on Image, Vision and Computing, Chongqing, China, 2018: 149–156. doi: 10.1109/ICIVC.2018.8492842.

杨燕, 王志伟. 基于均值不等关系优化的自适应图像去雾算法[J]. 电子与信息学报, 2020, 42(3): 755–763. doi: 10.11999/JEIT190368

YANG Yan and WANG Zhiwei. Adaptive image dehazing algorithm based on mean unequal relation optimization[J]. Journal of Electronics &Information Technology, 2020, 42(3): 755–763. doi: 10.11999/JEIT190368

JOHN J and WILSCY M. Enhancement of weather degraded video sequences using wavelet fusion[C]. The 7th IEEE International Conference on Cybernetic Intelligent Systems, London, England, 2008: 1–6. doi: 10.1109/UKRICIS.2008.4798926.

YOON I, KIM S, KIM D, et al. Adaptive defogging with color correction in the HSV color space for consumer surveillance system[J]. IEEE Transactions on Consumer Electronics, 2012, 58(1): 111–116. doi: 10.1109/TCE.2012.6170062

郭璠, 蔡自兴, 谢斌. 基于雾气理论的视频去雾算法[J]. 电子学报, 2011, 39(9): 2019–2025.

GUO Fan, CAI Zixing, and XIE Bin. Video defogging algorithm based on fog theory[J]. Acta Electronica Sinica, 2011, 39(9): 2019–2025.

刘海波, 杨杰, 吴正平, 等. 改进的基于雾气理论的视频去雾[J]. 光学精密工程, 2016, 24(7): 1789–1798. doi: 10.3788/OPE.20162407.1789

LIU Haibo, YANG Jie, WU Zhengping, et al. Improved video defogging based on fog theory[J]. Optics and Precision Engineering, 2016, 24(7): 1789–1798. doi: 10.3788/OPE.20162407.1789

马忠丽, 文杰, 郝亮亮. 海面舰船场景的视频图像海雾去除算法[J]. 系统工程与电子技术, 2014, 36(9): 1860–1867. doi: 10.3969/j.issn.1001-506X.2014.09.31

MA Zhongli, WEN Jie, and HAO Liangliang. Video image defogging algorithm for surface ship scenes[J]. Systems Engineering and Electronics, 2014, 36(9): 1860–1867. doi: 10.3969/j.issn.1001-506X.2014.09.31

LI Zhuwen, TAN Ping, TAN R T, et al. Simultaneous video defogging and stereo reconstruction[C]. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, USA, 2015: 4988–4997. doi: 10.1109/CVPR.2015.7299133.

BERMAN D, TREIBITZ T, AVIDAN S, et al. Non-local image dehazing[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, 2016: 1674–1682. doi: 10.1109/CVPR.2016.185.

KATO T, SHIMIZU I, and PAJDLA T. Selecting image pairs for SfM by introducing jaccard similarity[C]. The 15th IAPR International Conference on Machine Vision Applications (MVA), Nagoya, Japan, 2017: 25–29. doi: 10.23919/MVA.2017.7986764.

CAI Bolun, XU Xiangmin, and TAO Dacheng. Real-time video dehazing based on spatio-temporal MRF[C]. The 17th Pacific-Rim Conference on Multimedia on Advances in Multimedia Information Processing, Cham, 2016: 315–325. doi: 10.1007/978-3-319-48896-7_31.

ZHAO Dong, XU Long, YAN Yihua, et al. Multi-scale optimal fusion model for single image dehazing[J]. Signal Processing: Image Communication, 2019, 74: 253–265. doi: 10.1016/j.image.2019.02.004

YU Teng, SONG Kang, MIAO Pu, et al. Nighttime single image dehazing via pixel-wise alpha blending[J]. IEEE Access, 2019, 7: 114619–114630. doi: 10.1109/ACCESS.2019.2936049

施引文献

资源附件(0)

访问统计