Depth Map Error Concealment for 3D High Efficiency Video Coding
-
摘要: 基于多视点视频序列视点内、视点间存在的相关性，并结合视点间运动矢量共享技术，该文提出一种面向3维高效视频编码中深度序列传输丢包的错误隐藏算法。首先，根据3D高效视频编码(3D-HEVC)的分层B帧预测(HBP)结构和深度图纹理特征，将深度图丢失块分成运动块和静止块；然后，对于受损运动块，使用结合纹理结构的外边界匹配准则来选择相对最优的运动/视差矢量进行基于位移矢量补偿的错误掩盖，而对受损静止块采用参考帧直接拷贝进行快速错误隐藏；最后，使用参考帧拆分重组来获取新的运动/视差补偿块对修复质量较差的重建块进行质量提升。实验结果表明：相较于近年提出的对比算法，该文算法隐藏后的深度帧平均峰值信噪比(PSNR)能提升0.25～2.03 dB，结构相似度测量值(SSIM)能提升0.001～0.006，且修复区域的主观视觉质量与原始深度图更接近。Abstract: By using the intra-view and inter-view correlations and the motion vector-sharing, a depth map error concealment approach is proposed for 3D video coding based on the High Efficiency Video Coding (3D-HEVC) to combat the packet loss of the depth video transmission. Based on the Hierarchical B-frame Prediction (HBP) structure in 3D-HEVC and textured features of the depth map, all the lost coding units are firstly categorized into two classes, i.e., motion blocks and static blocks. Then, according to the outer boundary matching criterion combining the texture structure, the optimal motion/disparity vector is chosen for the damaged motion blocks to conduct the motion/disparity compensation based error concealment. Whereas, the direct copy is applied to concealling the damaged static blocks quickly. Finally, for the concealed blocks whose qualities are not ideal, the new motion/disparity compensation blocks reconstructed by the reference frames recombination are applied to improning the qualities of those blocks. The experimental results show that the repaired depth map concealed by the proposed approach can achieve 0.25～2.03 dB gain in term of the Peak-Signal-to-Noise Ratio (PSNR) and 0.001～0.006 gain in term of Structural Similarity Index Measure(SSIM). Moreover, the subjective visual quality of the repaired area is better in lines with the original depth maps.
-
表 1 组合参考块对应表
参考列表 参考帧号 L0 L1 R00 R01 … R0K R10 R11 … R1K L0 R00 P0 – – B00 B01 B0i B0K R01 – P1 – B10 B11 B1i B1K ∶ – – Pi Bi0 Bi1 Bii BiK R0K PK BK0 BK1 BKi BKK L1 R10 B00 B10 Bi0 BK0 PK+1 – – R11 B01 B11 Bi1 BK1 – PK+2 – ∶ B0i B1i Bii BKi – – PK+i R1K B0K B1K BiK BKK P2K 
下载: 导出CSV
表 2 QP=30, 25时各测试序列重建图像平均PSNR计算结果(dB)
QP 序列 原始 错误率50% 错误率25% 错误率10% 文献[6] 文献[7] MVC 本文算法 文献[6] 文献[7] MVC 本文算法 文献[6] 文献[7] MVC 本文算法 30 Kendo 42.761 36.406 36.122 35.894 37.103 37.896 37.359 36.786 38.310 39.449 38.925 38.025 39.833 Bookarrival 39.923 36.932 35.911 35.656 37.525 37.865 37.171 36.526 38.286 38.338 37.699 37.389 38.747 Poznanstreet 44.028 39.593 38.834 36.662 40.864 41.155 41.028 40.799 41.890 41.450 41.219 40.983 42.171 Balloon 40.574 38.713 38.512 37.691 39.269 39.514 39.456 38.347 39.839 39.706 39.638 38.485 39.958 25 Kendo 45.874 36.736 36.548 36.096 37.499 39.227 38.058 37.287 39.655 41.368 39.688 39.133 41.501 Bookarrival 41.625 37.981 36.733 36.354 38.270 38.411 37.488 36.947 39.921 39.728 39.196 38.463 40.187 Poznanstreet 46.122 39.955 38.928 37.066 41.561 41.723 41.178 40.802 42.319 42.113 41.797 41.207 42.622 Balloon 44.022 40.829 39.434 39.156 41.247 41.899 41.277 40.894 42.149 42.234 42.158 41.084 42.612
下载: 导出CSV
表 3 QP=30, 25时各测试序列重建图像平均SSIM计算结果
QP 序列 原始 错误率50% 错误率25% 错误率10% 文献[6] 文献[7] MVC 本文算法 文献[6] 文献[7] MVC 本文算法 文献[6] 文献[7] MVC 本文算法 30 Kendo 0.9867 0.9708 0.9688 0.9647 0.9721 0.9780 0.9776 0.9733 0.9785 0.9819 0.9816 0.9766 0.9822 Bookarrival 0.9565 0.9482 0.9457 0.9422 0.9495 0.9503 0.9481 0.9452 0.9510 0.9525 0.9518 0.9477 0.9534 Poznanstreet 0.9795 0.9732 0.9720 0.9714 0.9759 0.9746 0.9740 0.9738 0.9767 0.9751 0.9750 0.9744 0.9769 Balloon 0.9767 0.9738 0.9733 0.9720 0.9743 0.9745 0.9742 0.9736 0.9751 0.9747 0.9746 0.9743 0.9757 25 Kendo 0.9874 0.9722 0.9696 0.9660 0.9751 0.9789 0.9783 0.9765 0.9796 0.9828 0.9824 0.9795 0.9832 Bookarrival 0.9701 0.9577 0.9547 0.9539 0.9598 0.9634 0.9627 0.9613 0.9641 0.9643 0.9642 0.9633 0.9655 Poznanstreet 0.9878 0.9818 0.9803 0.9796 0.9824 0.9825 0.9821 0.9816 0.9848 0.9843 0.9841 0.9824 0.9849 Balloon 0.9855 0.9810 0.9803 0.9796 0.9821 0.9834 0.9833 0.9828 0.9839 0.9841 0.9836 0.9835 0.9844
下载: 导出CSV
-
LI Yufeng and CHEN Ruining. Motion vector recovery for video error concealment based on plane fitting[J]. Computer Engineering &Applications, 2018, 54(2): 182–187. doi: 10.3778/j.issn.1002-8331.1608-0085 XU Dawen and WANG Rangding. Two-dimensional reversible data hiding-based approach for intra-frame error concealment in H.264/AVC[J]. Journal of Visual Communication& Image Representation, 2016, 47(2): 369–379. doi: 10.1016/j.jvcir.2013.12.008 ZABIHI S M, GHANEI Y H, and MEHRSHAD N. Adaptive temporal error concealment method based on the MB behavior estimation in the video[C]. The 7th International Conference on Computer and Knowledge Engineering. Mashhad, Iran, 2017: 193–198. EL-SHAFAI W, EL-RABAIE S, EL-HALAWANY M M, et al. Recursive Bayesian filtering-based error concealment scheme for 3D video communication over severely lossy wireless channels[J]. Circuits Systems & Signal Processing, 2018(20): 1–32. doi: 10.1007/s00034-018-0786-8 周洋, 蒋刚毅, 郁梅, 等. 面向HBP编码格式的立体视频B帧整帧丢失分层错误隐藏算法[J]. 电子与信息学报, 2014, 36(2): 377–383. doi: 10.3724/SP.J.1146.2013.00528ZHOU Yang, JIANG Gangyi, YU Mei, et al. A hierarchical error concealment algorithm for entire B-frame loss in stereoscopic video coding with hierarchical B pictures[J]. Journal of Electronics &Information Technology, 2014, 36(2): 377–383. doi: 10.3724/SP.J.1146.2013.00528 SONG K, CHUNG T, OH Y, et al. Error concealment of multi-view video sequences using inter-view and intra-view correlations[J]. Journal of Visual Communication & Image Representation, 2009, 20(4): 281–292. doi: 10.1016/j.jvcir.2009.02.002 PENG Yantsung and COSMAN P C. Weighted boundary matching error concealment for HEVC using block partition decisions[C]. The 48th Asilomar Conference on Signals, Systems & Computers, Grove, USA, 2014: 921–925. doi: 10.1109/ACSSC.2014.7094587. KHATTAK S, MAUGEY T, HAMZAOUI R, et al. Temporal and inter-view consistent error concealment technique for multiview plus depth video[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2016, 26(5): 829–840. doi: 10.1109/TCSVT.2015.2418631 LIU Shujie, CHEN Ying, WANG Yekui, et al. Frame loss error concealment for multiview video coding[C]. IEEE International Symposium on Circuits and Systems (ISCAS 2008), Washington, USA, 2008: 3470–3473. ZHOU Yuan, XIANG Wei, and WANG Gengkun. Frame loss concealment for multiview video transmission over wireless multimedia sensor networks[J]. IEEE Sensors Journal, 2015, 15(3): 1892–1901. doi: 10.1109/JSEN.2014.2366511 HUANG Pincheng, LIN Jieru, LI Gwolong, et al. Improved depth-assisted error concealment algorithm for 3D video transmission[J]. IEEE Transactions on Multimedia, 2017, 19(11): 2625–2632. doi: 10.1109/TMM.2017.2694218 TAI Shenchuan, WANG Chuenching, HONG Chienshiang, et al. An efficient full frame algorithm for object-based error concealment in 3D depth-based video[J]. Multimedia Tools & Applications, 2016, 75(16): 9927–9947. doi: 10.1007/s11042-015-2899-4 时琳, 刘荣科, 李君烨. 基于深度信息的立体视频错误隐藏方法[J]. 电子与信息学报, 2012, 34(7): 1678–1684. doi: 10.3724/SP.J.1146.2011.01107SHI Ling, LIU Rongke, and LI Junye. Error concealment based on depth information for stereoscopic video coding[J]. Journal of Electronics &Information Technology, 2012, 34(7): 1678–1684. doi: 10.3724/SP.J.1146.2011.01107 YANG Meng, LAN Xuguang, ZHENG Nanning, et al. Depth-assisted temporal error concealment for intra frame slices in 3-D video[J]. IEEE Transactions on Broadcasting, 2014, 60(2): 385–393. doi: 10.1109/TBC.2014.2321676 YAN Bo and ZHOU Jie. Efficient frame concealment for depth image-based 3-D video transmission[J]. IEEE Transactions on Multimedia, 2012, 14(3): 936–941. doi: 10.1109/TMM.2012.2184743 MERKLE P, MULLER K, MARPE D, et al. Depth intra coding for 3D video based on geometric primitives[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2016, 26(3): 570–582. doi: 10.1109/TCSVT.2015.2407791 HEWAGE C T E R, WORRALL S, DOGAN S, et al. Frame concealment algorithm for stereoscopic video using motion vector sharing[C]. 2008 IEEE International Conference on Multimedia and Expo, Hannover, Germany, 2008: 485–488. doi: 10.1109/ICME.2008.4607477. ASSUNCAO P, MARCELINO S, SOARES S, et al. Spatial error concealment for intra-coded depth maps in multiview video-plus-depth[J]. Multimedia Tools &Applications, 2017, 76(12): 13835–13858. doi: 10.1007/s11042-016-3766-7 HTM Reference Software 14.0. http://hevc.kw.bbc.co.uk/git/w/jctvc-3de.git, 2016. WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600–612. doi: 10.3969/j.issn.1006-8961.2006.12.00.003 -
图(5) / 表(4)
计量
- 文章访问数: 2182
- HTML全文浏览量: 1011
- PDF下载量: 43
- 被引次数: 0
下载:
