基于局部亮度直方图的自适应视频帧类型决策算法

刘鹏宇; 张悦; 贾克斌; 段堃; 刘畅; 孙萱; 崔腾鹤

doi:10.11999/JEIT211199

基于局部亮度直方图的自适应视频帧类型决策算法

doi: 10.11999/JEIT211199

1.
北京工业大学信息学部北京 100124
2.
先进信息网络北京实验室北京 100124
3.
计算机智能与智能系统北京市重点实验室北京 100124

基金项目: 国家重点研发计划(2018YFF01010100)，北京自然科学基金(4212001)，青海省重点研发与转化计划(2022-QY-205)

详细信息

作者简介:
刘鹏宇：女，副教授，博士生导师，研究方向为多媒体信息处理

张悦：女，硕士生，研究方向为视频编码技术

贾克斌：男，教授，博士生导师，研究方向为信息与通信系统

段堃：男，硕士，研究方向为视频编码技术

刘畅：女，博士生，研究方向为3D视频编码

孙萱：男，硕士生，研究方向为视频编码技术

崔腾鹤：男，硕士，研究方向为视频编码技术

通讯作者:
刘鹏宇　liupengyu@bjut.edu.cn

中图分类号: TN919.81
计量
- 文章访问数: 510
- HTML全文浏览量: 366
- PDF下载量: 77
- 被引次数: 0
出版历程
- 收稿日期: 2021-11-01
- 修回日期: 2022-03-25
- 网络出版日期: 2022-04-15
- 刊出日期: 2023-01-17

Adaptive Video Frame Type Decision Algorithm Based on Local Luminance Histogram

1.
The Information Department, Beijing University of Technology, Beijing 100124, China
2.
Advanced Information Network Beijing Laboratory, Beijing 100124, China
3.
Computational Intelligence and Intelligent Systems Beijing Key Laboratory, Beijing 100124, China

Funds: The National Key Research and Development Program of China (2018YFF01010100), The Beijing Natural Science Foundation (4212001), The Key R&D and Transformation Program of Qinghai Province (2022-QY-205)

摘要

摘要: 视频帧类型决策是影响视频编码效率的关键因素之一。为提升x265视频编码器的编码性能，该文提出基于局部亮度直方图的自适应视频帧类型决策算法。首先，在64×64大小的编码树单元(CTU)级别上统计各帧局部亮度直方图，用帧间局部亮度直方图差异表征帧间场景变换程度；其次，引入帧内编码帧(I帧)检测窗，在检测窗内通过比较帧间场景变换程度自适应确定I帧；最后，根据帧间场景变换程度与迷你图像组(MiniGOP)大小之间的相关性确定MiniGOP大小，从而自适应确定普通P和B帧(GPB帧)及双向预测编码帧(B帧)。实验结果表明，与x265标准中的相关算法相比，所提算法能够有效降低x265的编码复杂度，可在减少近5%编码时间的前提下，实现视频I帧、GPB帧和B帧的高效自适应决策。
- 视频编码 /
- x265 /
- 局部亮度直方图 /
- 视频帧类型决策
Abstract: The video frame type decision is one of the key factors affecting the efficiency of video coding. This paper proposes an adaptive video frame type decision algorithm based on local luminance histogram to improve the x265 encoding performance. Firstly, the local luminance histograms of frames are calculated at the level of 64×64 Coding Tree Unit (CTU), and the difference of local luminance histogram between frames is used to represent the degree of scene variation between frames. Secondly, Intra-coded picture (I-frame) detection window is introduced. I-frame is determined by comparing the degree of scene variation between frames. Finally, the Mini Group Of Picture (MiniGOP) size is determined according to the correlation between the degree of scene variation and MiniGOP size, so as to determine adaptively Generalized P and B picture (GPB-frame) and Bidirectionally predicted picture (B-frame). Experimental results show that compared with the relevant algorithms in x265, the proposed algorithm can effectively reduce the coding complexity of x265, and decide I/GPB/B-frame efficiently and adaptively with nearly 5% less coding time.
- Video coding /
- x265 /
- Local luminance histogram /
- Video frame type decision

HTML全文

图 1 GOP示意图

下载: 全尺寸图片幻灯片

图 2 基于局部亮度直方图的自适应视频帧类型决策算法流程图

下载: 全尺寸图片幻灯片

图 3 某场景切换处前后两帧全局亮度直方图和CTU局部亮度直方图

下载: 全尺寸图片幻灯片

图 4 I帧检测过程

下载: 全尺寸图片幻灯片

图 5 MiniGOP示意图

下载: 全尺寸图片幻灯片

表 1 全局亮度直方图差异$ {D_{{\text{hist}}}} $与局部亮度直方图差异$ {D_{{\text{local\_hist}}}} $比较

序列	总帧数	场景切换数	$ {D_{{\text{hist}}}} > {D_{{\text{local\_hist}}}} $的场景切换数	${D_{ {\text{hist} } } } < {D_{ {\text{local\_hist} } } }$的场景切换数
Kimono	240	1	0	1
送你一朵小红花	4089	92	0	92
战狼2	4423	185	0	185
F1赛车越野	1880	48	0	0

下载: 导出CSV

表 2 本场景切换检测算法与x265和参考文献[11]的检测准确度对比

序列	分辨率	场景切换数	x265		参考文献[11]		本文算法
序列	分辨率	场景切换数	P(%)	R(%)	P(%)	R(%)	P(%)	R(%)
Traffic	2560×1600	0	100	100	100	100	100	100
Kimono	1920×1080	1	100	100	100	100	100	100
FourPeople	1280×720	0	100	100	100	100	100	100
BQMall	832×480	0	100	100	100	100	100	100
RaceHorses	416×240	0	100	100	100	100	100	100
送你一朵小红花	1920×1056	92	89.39	64.13	76.19	52.17	100	81.52
战狼2	1280×720	185	96.25	83.24	71.88	37.30	97.67	90.81
红海行动	1280×720	131	89.36	32.06	72.97	41.22	95.54	81.68
变形金刚5	1280×720	52	93.75	57.69	73.53	48.08	95.83	88.46
F1赛车越野	960×540	48	86.84	68.75	74.19	47.92	93.33	87.50

下载: 导出CSV

表 3 本场景切换检测与x265算法运行耗时对比(μs)

序列	分辨率	x265	本文算法
Traffic	2560×1600	46071.36	34.13
Kimono	1920×1080	17382.75	12.77
FourPeople	1280×720	6774.90	4.93
BQMall	832×480	2069.15	1.46
RaceHorses	416×240	662.11	0.92

下载: 导出CSV

表 4 本文算法与x265中两种算法和参考文献[13]的算法性能比较

Class	序列	x265快速		x265 Viterbi		参考文献[13]		本文算法
Class	序列	BDPSNR	$ \Delta T $(%)	BDPSNR	$ \Delta T $(%)	BDPSNR	$ \Delta T $(%)	BDPSNR	$ \Delta T $(%)
Class A	Traffic	2.618	1.623	–1.236	2.513	0.605	1.59	0.524	–1.170
Class A	PeopleOnStreet	0.139	0.510	–2.195	3.240	0.020	1.07	–1.056	–0.532
Class B	Cactus	5.742	2.681	–0.615	6.368	0.361	0.23	3.892	0.216
Class B	Kimono	–1.242	1.592	–2.262	2.425	0.115	–3.38	–1.369	–1.579
Class C	BasketballDirll	4.167	2.412	0.106	7.592	0.632	1.25	–1.315	0.822
Class C	PartyScene	8.163	6.423	–1.193	4.715	0.553	0.84	2.125	0.583
Class D	BasketballPass	1.203	4.741	–2.007	4.23	0.487	5.27	1.773	–0.836
Class D	RaceHorses	–0.276	4.285	–1.053	9.542	0.196	1.75	–1.741	–1.474
Class E	FourPeople	3.658	2.130	1.507	2.776	0.951	2.89	–3.773	–0.256
Class E	vidyo4	1.646	3.359	2.119	2.034	0.622	2.45	–1.410	–0.572
Class F	SlideShow	–0.892	3.824	–6.364	5.186	1.494	1.59	–9.727	–0.119
Class F	SlideEditing	–0.672	2.535	–7.382	8.571	0.417	2.15	–10.631	–1.658
平均值		2.021	3.010	–1.715	4.933	0.537	1.475	–1.892	–0.548

下载: 导出CSV

参考文献(20)

[1]	JCT-VC. ITU-T Recommendation H. 265 High efficiency video coding[S]. Geneva: ITU–T, 2013.
[2]	CORREA G, ASSUNCAO P, AGOSTINI L, et al. Performance and computational complexity assessment of high-efficiency video encoders[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 22(12): 1899–1909. doi: 10.1109/TCSVT.2012.2223411
[3]	BOSSEN F, BROSS B, SUHRING K, et al. HEVC complexity and implementation analysis[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 22(12): 1685–1696. doi: 10.1109/TCSVT.2012.2221255
[4]	GAO Yuan, LIU Pengyu, WU Yueying, et al. Quadtree degeneration for HEVC[J]. IEEE Transactions on Multimedia, 2016, 18(12): 2321–2330. doi: 10.1109/TMM.2016.2598481
[5]	ESAKKI G, PANAYIDES A, TEEPARTHI, et al. A comparative performance evaluation of VP9, x265, SVT-AV1, VVC codecs leveraging the VMAF perceptual quality metric[J]. SPIE, 2020: 1151010.
[6]	x265 ORG. x265 HEVC encoder[EB/OL]. http://www.x265.org/, 2017.
[7]	段堃. 基于H. 265/HEVC的高压缩率与低复杂度编码技术研究[D]. [硕士论文], 北京工业大学, 2020. DUAN Kun. Research on hight compression rate and low complexity coding based on H. 265/HEVC[D]. [Master dissertation], Beijing University of Technology, 2020.
[8]	FUAD M, ERNAWAN F, and HUI L J. Video scene change detection based on histogram analysis for hiding message[J]. Journal of Physics:Conference Series, 2021, 1918(4): 042141. doi: 10.1088/1742-6596/1918/4/042141
[9]	KANG S J. Positional analysis-based scene-change detection algorithm[C]. Proceedings of 2015 IEEE International Conference on Consumer Electronics, Las Vegas, USA, 2015: 11–12.
[10]	CHO S I and KANG S J. Histogram shape-based scene-change detection algorithm[J]. IEEE Access, 2019, 7: 27662–27667. doi: 10.1109/ACCESS.2019.2898889
[11]	刘辉, 刘立程, 郝禄国, 等. 一种场景切换下的HEVC码率控制算法[J]. 电视技术, 2017, 41(6): 1–5. doi: 10.16280/j.videoe.2017.06.001 LIU Hui, LIU Licheng, HAO Luguo, et al. A rate control algorithm for HEVC based on scene change[J]. Video Engineering, 2017, 41(6): 1–5. doi: 10.16280/j.videoe.2017.06.001
[12]	VideoLAN ORG. x264, the best H. 264/AVC encoder[EB/OL]. https://www.videolan.org/developers/x264.html, 2013.
[13]	LIU Zhenyu, WANG Libo, LI Xiaobo, et al. Optimize x265 rate control: An exploration of lookahead in frame bit allocation and slice type decision[J]. IEEE Transactions on Image Processing, 2019, 28(5): 2558–2573. doi: 10.1109/TIP.2018.2887200
[14]	FORNEY G D. The viterbi algorithm[J]. Proceedings of the IEEE, 1973, 61(3): 268–278. doi: 10.1109/PROC.1973.9030
[15]	《送你一朵小红花》“珍惜版”预告导演韩延开启“生命三部曲”[EB/OL]. https://www.mgtv.com/b/348435/10455525.html?fpa=se&lastp=so_result, 2020. A Little Red Flower. Clip: “Trilogy of Life” directed by Han Yan[EB/OL]. https://www.mgtv.com/b/348435/10455525.html?fpa=se&lastp=so_result, 2020.
[16]	《战狼2》片段: 冷锋Rachel飙车智斗“雇佣兵”[EB/OL]. https://www.mgtv.com/l/100009562/4052086.html?lastp=so_result, 2017. Wolf Warriors 2. Clip: Leng Feng and Rachel compete with mercenaries[EB/OL]. https://www.mgtv.com/l/100009562/4052086.html?lastp=so_result, 2017.
[17]	见过F1赛车越野吗? 红牛是真敢作, 2亿的车就这么糟蹋?[EB/OL]. https://v.youku.com/v_show/id_XMzQzNjQ1MjU1Mg==.html?spm=a2h0c.8166622.PhoneSokuUgc_2.dtitle, 2018. Have you ever seen an F1 car off-road?[EB/OL]. https://v.youku.com/v_show/id_XMzQzNjQ1MjU1Mg==.html?spm=a2h0c.8166622.PhoneSokuUgc_2.dtitle, 2018.
[18]	《红海行动》“蛟龙逆袭”版预告蛟龙突击队展现超强作战力[EB/OL]. https://www.mgtv.com/b/315515/4293086.html?fpa=se&lastp=so_result, 2018. "Operation Red Sea" clip: Preview version of "Jiaolong Attack"[EB/OL]. https://www.mgtv.com/b/315515/4293086.html?fpa=se&lastp=so_result, 2018.
[19]	《变形金刚5》并肩作战预告人类联手变形金刚拯救地球[EB/OL]. https://www.mgtv.com/b/308889/3994088.html?fpa=se&lastp=so_result, 2017. Transformers 5 clip: Side by side trailer[EB/OL]. https://www.mgtv.com/b/308889/3994088.html?fpa=se&lastp=so_result, 2017.
[20]	BJØNTEGAARD G. Calculation of average PSNR differences between RD-curves[C]. Proceedings of the 13th Video Coding Experts Group Meeting, Austin, USA, 2001: 290–291.