Fast Partition Algorithm in Depth Map Intra-frame Coding Unit Based on Multi-branch Network

LIU Chang; JIA Kebin; LIU Pengyu

doi:10.11999/JEIT211010

Volume 44 Issue 12

Dec. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(12): 4357-4366

LIU Chang, JIA Kebin, LIU Pengyu. Fast Partition Algorithm in Depth Map Intra-frame Coding Unit Based on Multi-branch Network[J]. Journal of Electronics & Information Technology, 2022, 44(12): 4357-4366. doi: 10.11999/JEIT211010

Citation:

LIU Chang, JIA Kebin, LIU Pengyu. Fast Partition Algorithm in Depth Map Intra-frame Coding Unit Based on Multi-branch Network[J]. Journal of Electronics & Information Technology, 2022, 44(12): 4357-4366. doi: 10.11999/JEIT211010

Citation:

PDF( 3726 KB)

Fast Partition Algorithm in Depth Map Intra-frame Coding Unit Based on Multi-branch Network

doi: 10.11999/JEIT211010

1.
Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
2.
Beijing Laboratory of Advanced Information Networks, Beijing 100124, China
3.
Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing 100124, China

Funds: The National Key Research and Development Project of China (2018YFF01010100), Beijing Natural Science Foundation (4212001), The Basic Research Program of Qinghai Province (2020-ZJ-709, 2021-ZJ-704)

Received Date: 2021-09-23
Accepted Date: 2021-12-06
Rev Recd Date: 2021-12-01

Available Online: 2021-12-11

Publish Date: 2022-12-16

Abstract

Abstract

Three Dimensional-High Efficiency Video Coding (3D-HEVC) standard is the latest Three-Dimensional (3D) video coding standard, but the coding complexity increases greatly due to the introduction of depth map coding technology. Among them, the quad-tree partition of depth map intra-frame Coding Unit (CU) accounts for more than 90% of the coding complexity in 3D-HEVC. Therefore, for the intra-frame coding of depth map in 3D-HEVC, considering the high complexity of CU quad-tree partition, a fast prediction scheme of CU partition structure based on deep learning is proposed. Firstly, the dataset of CU partition structure information for learning depth map is constructed. Secondly, a Multi-Branch Convolutional Neural Network (MB-CNN) model for predicting the CU partition structure is built. Then, the MB-CNN model is trained by using the built dataset. Finally, the MB-CNN model is embedded into the 3D-HEVC test platform, which reduces greatly the complexity of CU partition by predicting the partition structure of CU in depth map intra-frame coding. Experimental results show that the proposed algorithm reduces effectively the coding complexity of 3D-HEVC without significant synthesized view quality distortion. Specifically, compared to the standard method, the coding complexity on the standard test sequence is reduced by 37.4%.
- Three Dimensional-High Efficiency Video Coding(3D-HEVC),
- Depth map,
- Intra-frame coding,
- Coding Unit (CU) partition,
- Deep learning

FullText(HTML)

References(22)

References

[1]	LIU Shan, LIU Lu, YANG Hua, et al. Research on 5G technology based on Internet of things[C]. 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China, 2020: 1821–1823.
[2]	KUFA J and KRATOCHVIL T. Visual quality assessment considering ultra HD, Full HD resolution and viewing distance[C]. The 29th International Conference Radioelektronika, Pardubice, Czech Republic, 2019: 1–4.
[3]	LI Tiansong, YU Li, WANG Hongkui, et al. A bit allocation method based on inter-view dependency and spatio-temporal correlation for multi-view texture video coding[J]. IEEE Transactions on Broadcasting, 2021, 67(1): 159–173. doi: 10.1109/TBC.2020.3028340
[4]	王莉, 曹一凡, 杜高明, 等. 一种低延迟的3维高效视频编码中深度建模模式编码器[J]. 电子与信息学报, 2019, 41(7): 1625–1632. doi: 10.11999/JEIT180798 WANG Li, CAO Yifan, DU Gaoming, et al. A low-latency depth modelling mode-1 encoder in 3D-high efficiency video coding standard[J]. Journal of Electronics &Information Technology, 2019, 41(7): 1625–1632. doi: 10.11999/JEIT180798
[5]	CHEN Ying, HANNUKSELA M M, SUZUKI T, et al. Overview of the MVC + D 3D video coding standard[J]. Journal of Visual Communication and Image Representation, 2014, 25(4): 679–688. doi: 10.1016/j.jvcir.2013.03.013
[6]	TIAN Shishun, ZHANG Lu, ZOU Wenbin, et al. Quality assessment of DIBR-synthesized views: An overview[J]. Neurocomputing, 2021, 423: 158–178. doi: 10.1016/j.neucom.2020.09.062
[7]	齐美彬, 陈秀丽, 杨艳芳, 等. 高效率视频编码帧内预测编码单元划分快速算法[J]. 电子与信息学报, 2014, 36(7): 1699–1705. doi: 10.3724/SP.J.1146.2013.01148 QI Meibin, CHEN Xiuli, YANG Yanfang, et al. Fast coding unit splitting algorithm for high efficiency video coding intra prediction[J]. Journal of Electronics &Information Technology, 2014, 36(7): 1699–1705. doi: 10.3724/SP.J.1146.2013.01148
[8]	ZUO Jiabao, CHEN Jing, ZENG Huanqiang, et al. Bi-layer texture discriminant fast depth intra coding for 3D-HEVC[J]. IEEE Access, 2019, 7: 34265–34274. doi: 10.1109/ACCESS.2019.2897161
[9]	LI Tiansong, WANG Hongkui, CHEN Yamei, et al. Fast depth intra coding based on spatial correlation and rate distortion cost in 3D-HEVC[J]. Signal Processing:Image Communication, 2020, 80: 115668. doi: 10.1016/j.image.2019.115668
[10]	LI Tiansong, YU Li, WANG Shengwei, et al. Simplified depth intra coding based on texture feature and spatial correlation in 3D-HEVC[C]. 2018 Data Compression Conference, Snowbird, USA, 2018: 421.
[11]	SALDANHA M, SANCHEZ G, MARCON C, et al. Fast 3D-HEVC depth map encoding using machine learning[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(3): 850–861. doi: 10.1109/TCSVT.2019.2898122
[12]	FU Changhong, CHEN Hao, CHAN Y L, et al. Fast depth intra coding based on decision tree in 3D-HEVC[J]. IEEE Access, 2019, 7: 173138–173147. doi: 10.1109/ACCESS.2019.2956994
[13]	SALDANHA M, SANCHEZ G, MARCON C, et al. Fast 3D-HEVC depth maps intra-frame prediction using data mining[C]. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, Canada, 2018: 1738–1742.
[14]	XU Mai, LI Tianyi, WANG Zulin, et al. Reducing complexity of HEVC: A deep learning approach[J]. IEEE Transactions on Image Processing, 2018, 27(10): 5044–5059. doi: 10.1109/TIP.2018.2847035
[15]	TANG Genwei, JING Minge, ZENG Xiaoyang, et al. Adaptive CU split decision with pooling-variable CNN for VVC intra encoding[C]. 2019 IEEE Visual Communications and Image Processing (VCIP), Sydney, Australia, 2019: 1–4.
[16]	李雅婷, 杨静. 3D-HEVC深度图帧内预测快速编码算法[J]. 光电子·激光, 2020, 31(2): 222–228. doi: 10.16136/j.joel.2020.02.0344 LI Yating and YANG Jing. Fast intra coding algorithm for depth map in 3D-HEVC[J]. Journal of Optoelectronics Laser, 2020, 31(2): 222–228. doi: 10.16136/j.joel.2020.02.0344
[17]	XIE Saining and TU Zhuowen. Holistically-nested edge detection[J]. International Journal of Computer Vision, 2017, 125(1/3): 3–18. doi: 10.1007/s11263-017-1004-z
[18]	SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. Proceedings of the 3rd International Conference on Learning Representations (ICLR), San Diego, USA, 2015: 1–14.
[19]	Tanimoto Lab. Nagoya University multi-view sequences download list[EB/OL].https://www.fujii.nuee.nagoya-u.ac.jp/multiview-data/, 2017.
[20]	FENG Zeqi, LIU Pengyu, JIA Kebin, et al. Fast intra CTU depth decision for HEVC[J]. IEEE Access, 2018, 6: 45262–45269. doi: 10.1109/ACCESS.2018.2864881
[21]	JCT-3V. 3D-HEVC reference software[EB/OL]. https://mpeg.chiariglione.org/standards/mpeg-h/hevc-reference-software.
[22]	BJONTEGAARD G. Calculation of average PSNR differences between RD curves[C]. The 13th Video Coding Experts Group Meeting, Austin, USA, 2001: VCEG-M33.