No-reference Image Quality Evaluation for Multiply-distorted Images Based on Spatial Domain Coding
-
摘要: 针对难以准确有效地提取混合失真图像质量特征的问题，该文提出一种基于空间分布分析的图像质量评价方法。首先将图像进行亮度系数归一化处理，然后将图像进行分块，利用卷积神经网络(CNN)进行端对端的深度学习，采用多层次卷积核堆叠的方法获取图像的质量感知特征，并通过全连接层将特征映射到图像块的质量分数。再将块质量分数汇总获取质量池，通过对质量池中局部质量的空间分布情况进行分析，提取能够表征其空间分布情况的特征，然后采用神经网络建立局部质量到整体质量的映射模型，将图像的局部质量进行汇总。最后在MLIVE, MDID2013, MDID2016混合失真图像库中进行性能测试以及与相关的对比算法进行比较，验证了该算法的有效性。Abstract: Considering the problem that it is difficult to accurately and effectively extract the quality features of mixed distortion image, an image quality assessment method based on spatial distribution analysis is proposed. Firstly, the brightness coefficients of the image are normalized, and the image is divided into blocks. While the Convolutional Neural Network (CNN) is used for end-to-end depth learning, the multi-level stacking of convolution cores is applied to acquire image quality perception features. The feature is mapped to the mass fraction of the image block through the full connection layer, then the quality pool is obtained by aggregating the quality of the block. Through the analysis of the spatial distribution of local quality in the quality pool, the features that can represent its spatial distribution are extracted, and then the mapping model from local quality to overall quality is established by the neural network to aggregate the local quality of the image. Finally, the effectiveness of the algorithm is verified by the performance tests in MLIVE, MDID2013 and MDID2016 mixed distortion image databases.
-
表 1 混合失真图像库描述
图像库 参考图像 失真类型 图像数 主观评分 MLIVE 15 模糊+噪声/模糊+JPEG压缩 450 0-100(DMOS) MDID2013 12 模糊+噪声+JPEG压缩 324 0-1(DMOS) MDID2016 20 模糊+噪声+对比度+JPEG压缩+JP2K压缩 1600 0-8(MOS) 
下载: 导出CSV
表 2 MLIVE图像库中各特征性有效性实验
特征 PLCC SROCC KROCC 均值 0.951 0.941 0.753 方差 0.795 0.740 0.625 偏斜度 0.570 0.472 0.334 峰度 0.461 0.493 0.348 整体评价 0.961 0.951 0.781
下载: 导出CSV
表 3 不同图像库中算法性能测试
图像库 PLCC SROCC KROCC RMSE MLIVE(Part1) 0.969 0.956 0.822 4.502 MLIVE(Part2) 0.957 0.942 0.784 4.944 MLIVE(All) 0.961 0.951 0.781 4.831 MDID2013 0.935 0.922 0.755 0.017 MDID2016 0.921 0.917 0.749 0.756
下载: 导出CSV
表 4 算法性能对比实验
算法 MLIVE(450 images) MDID2013(324 images) PLCC SROCC RMSE PLCC SROCC RMSE PSNR FR 0.740 0.677 12.724 0.561 0.560 0.042 SSIM FR 0.926 0.902 6.797 0.457 0.450 0.045 VIF[18] FR 0.932 0.915 6.761 0.915 0.905 0.020 BRISQUE[5] NR 0.924 0.900 7.143 0.833 0.819 0.027 NFERM[6] NR 0.917 0.898 7.459 0.871 0.855 0.024 GWH-LBP[7] NR 0.949 0.944 8.873 0.913 0.908 0.019 HOSA[8] NR 0.926 0.902 6.974 0.892 0.872 0.021 Zhou[10] NR 0.951 0.943 5.747 0.919 0.907 0.019 CORNIA[11] NR 0.916 0.900 7.586 0.904 0.898 0.020 NIQE[19] NR 0.839 0.775 10.294 0.563 0.545 0.042 SISBLM[20] NR 0.895 0.878 8.439 0.814 0.808 0.030 本文算法 NR 0.961 0.951 4.831 0.935 0.922 0.017
下载: 导出CSV
-
GU Ke, TAO Dacheng, QIAO Junfei, et al. Learning a no-reference quality assessment model of enhanced images with big data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(4): 1301–1313. doi: 10.1109/TNNLS.2017.2649101 FREITAS P G, AKAMINE W Y L, and FARIAS M C Q. No-Reference image quality assessment using orthogonal color planes patterns[J]. IEEE Transactions on Multimedia, 2018, 20(12): 3353–3360. doi: 10.1109/TMM.2018.2839529 张敏辉, 杨剑. 评价SAR图像去噪效果的无参考图像质量指标[J]. 重庆邮电大学学报: 自然科学版, 2018, 30(4): 530–536. doi: 10.3979/j.issn.1673-825X.2018.04.014ZHANG Minhui and YANG Jian. A new referenceless image quality index to evaluate denoising performance of SAR images[J]. Journal of Chongqing University of Posts and Telecommunications:Natural Science Edition, 2018, 30(4): 530–536. doi: 10.3979/j.issn.1673-825X.2018.04.014 徐弦秋, 刘宏清, 黎勇, 等. 基于RGB通道下模糊核估计的图像去模糊[J]. 重庆邮电大学学报: 自然科学版, 2018, 30(2): 216–221. doi: 10.3979/j.issn.1673-825X.2018.02.009XU Xianqiu, LIU Hongqing, LI Yong, et al. Image deblurring with blur kernel estimation in RGB channels[J]. Journal of Chongqing University of Posts and Telecommunications:Natural Science Edition, 2018, 30(2): 216–221. doi: 10.3979/j.issn.1673-825X.2018.02.009 MITTAL A, MOORTHY A K, and BOVIK A C. No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing, 2012, 21(12): 4695–4708. doi: 10.1109/TIP.2012.2214050 GU Ke, ZHAI Guangtao, YANG Xiaokang, et al. Using free energy principle for blind image quality assessment[J]. IEEE Transactions on Multimedia, 2015, 17(1): 50–63. doi: 10.1109/TMM.2014.2373812 LI Qiaohong, LIN Weisi, and FANG Yuming. No-reference quality assessment for multiply-distorted images in gradient domain[J]. IEEE Signal Processing Letters, 2016, 23(4): 541–545. doi: 10.1109/LSP.2016.2537321 DAI Tao, GU Ke, NIU Li, et al. Referenceless quality metric of multiply-distorted images based on structural degradation[J]. Neurocomputing, 2018, 290: 185–195. doi: 10.1016/j.neucom.2018.02.050 JIA Sen and ZHANG Yang. Saliency-based deep convolutional neural network for no-reference image quality assessment[J]. Multimedia Tools and Applications, 2018, 77(12): 14859–14872. doi: 10.1007/s11042-017-5070-6 ZHOU Wujie, YU Lu, QIAN Yaguan, et al. Deep blind quality evaluator for multiply distorted images based on monogenic binary coding[J]. Journal of Visual Communication and Image Representation, 2019, 60: 305–311. doi: 10.1016/j.jvcir.2019.03.001 YE Peng, KUMAR J, KANG Le, et al. Unsupervised feature learning framework for no-reference image quality assessment[C]. 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, USA, 2012: 1098–1105. doi: 10.1109/CVPR.2012.6247789. BOUREAU Y L, BACH F, LECUN Y, et al. Learning mid-level features for recognition[C]. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, USA, 2010: 2559–2566. doi: 10.1109/CVPR.2010.5539963. 孙娅楠, 林文斌. 梯度下降法在机器学习中的应用[J]. 苏州科技大学学报: 自然科学版, 2018, 35(2): 26–31. doi: 10.12084/j.issn.2096-3289.2018.02.006SUN Yanan and LIN Wenbin. Application of gradient descent method in machine learning[J]. Journal of Suzhou University of Science and Technology:Natural Science, 2018, 35(2): 26–31. doi: 10.12084/j.issn.2096-3289.2018.02.006 JAYARAMAN D, MITTAL A, MOORTHY A K, et al. Objective quality assessment of multiply distorted images[C]. 2012 Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers, Pacific Grove, USA, 2012: 1693–1697. doi: 10.1109/ACSSC.2012.6489321. GU Ke, ZHAI Guangtao, YANG Xiaokang, et al. Hybrid no-reference quality metric for singly and multiply distorted images[J]. IEEE Transactions on Broadcasting, 2014, 60(3): 555–567. doi: 10.1109/TBC.2014.2344471 SUN Wen, ZHOU Fei, and LIAO Qingmin. MDID: A multiply distorted image database for image quality assessment[J]. Pattern Recognition, 2017, 61: 153–168. doi: 10.1016/j.patcog.2016.07.033 ZHANG Min, MURAMATSU C, ZHOU Xiangrong, et al. Blind image quality assessment using the joint statistics of generalized local binary pattern[J]. IEEE Signal Processing Letters, 2015, 22(2): 207–210. doi: 10.1109/LSP.2014.2326399 SHEIKH H R and BOVIK A C. Image information and visual quality[J]. IEEE Transactions on Image Processing, 2006, 15(2): 430–444. doi: 10.1109/TIP.2005.859378 MITTAL A, SOUNDARARAJAN R, and BOVIK A C. Making a "completely blind" image quality analyzer[J]. IEEE Signal Processing Letters, 2013, 20(3): 209–212. doi: 10.1109/LSP.2012.2227726 LI Qiaohong, LIN Weisi, XU Jingtao, et al. Blind image quality assessment using statistical structural and luminance features[J]. IEEE Transactions on Multimedia, 2016, 18(12): 2457–2469. doi: 10.1109/TMM.2016.2601028 -
下载:
图(6) / 表(5)
计量
- 文章访问数: 2830
- HTML全文浏览量: 880
- PDF下载量: 100
- 被引次数: 0
