基于区域协方差的视频显著度局部空时优化模型

田畅; 姜青竹; 吴泽民; 刘涛; 胡磊

doi:10.11999/JEIT151122

基于区域协方差的视频显著度局部空时优化模型

doi: 10.11999/JEIT151122

基金项目:

国家自然科学基金青年基金(61501509)

计量
- 文章访问数: 1387
- HTML全文浏览量: 144
- PDF下载量: 665
- 被引次数: 0
出版历程
- 收稿日期: 2015-10-08
- 修回日期: 2016-02-29
- 刊出日期: 2016-07-19

A Local Spatiotemporal Optimization Framework for Video Saliency Detection Using Region Covariance

Funds:

The National Natural Science Youth Foundation of China (61501509)

摘要

摘要: 显著度检测在计算机视觉中应用非常广泛，图像级的显著度检测研究已较为成熟，但视频显著度因其高度挑战性研究相对较少。该文借鉴图像级显著度算法的思想，提出一种通用的空时特征提取与优化模型来检测视频显著度。首先利用区域协方差矩阵构造视频的空时特征描述子，然后计算对比度得出初始显著图，最后通过联合前后帧的局部空时优化模型得到最终的显著图。在2个公开视频显著性数据集上的实验结果表明，所提算法性能优于目前的主流算法，同时具有良好的扩展性。
- 视频显著度 /
- 区域协方差 /
- 局部对比度 /
- 局部空时优化
Abstract: Visual saliency is widely applied to computer vision. Image saliency detection has been extensively studied, while there are only a few effective methods of computing saliency for videos owing to its high challenge. Inspired by image saliency methods, this paper proposes a unified spatiotemporal feature extraction and optimization framework for video saliency. First, the spatiotemporal feature descriptor is constructed via region covariance. Then, initial saliency map is computed by the local contrast of the descriptor. Finally, a local spatiotemporal optimization framework considering the previous and next frames of the current one is modeled to obtain the final saliency map. Extensive experiments on two public datasets demonstrate that the proposed algorithm not only outperforms the state-of-the-art methods, but also is of great extendibility.
- Video saliency /
- Region covariance /
- Local contrast /
- Local spatiotemporal optimization

HTML全文

参考文献(33)

BORJI A, CHENG M, JIANG H, et al. Salient object detection: A benchmark[J]. IEEE Transactions on Image Processing, 2015, 24(12): 5706-5722. doi: 10.1109/ TIP.2015.2487833.

ROTHER C, KOLMOGOROV V, and BLAKE A. Grabcut: Interactive foreground extraction using iterated graph cuts[J]. ACM Transactions on Graphics, 2004, 23(1): 309-314. doi: 10.1145/1186562.1015720.

DING Y, XIAO J, and YU J. Importance filtering for image retargeting[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, 2011: 89-96. doi: 10.1109/CVPR.2011.5995445.

MAHADEVAN V and VASOONCEIOS N. Saliency-based discriminant tracking[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, USA, 2009: 1007-1013. doi: 10.1109/CVPR.2009.5206573.

SHARMA G, JURIE F, and SCHMID C. Discriminative spatial saliency for image classification[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Rhode Island, USA, 2012: 3506-3513. doi: 10.1109/ CVPR.2012.6248093.

HADIZADEH H and BAJIC I. Saliency-aware video compression[J]. IEEE Transactions on Image Processing, 2014, 23(1): 19-33. doi: 10.1109/TIP.2013.2282897.

CHENG M, ZHANG G, MIERA N, et al. Global contrast based salient region detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, 2011: 409-416. doi: 10.1109/ CVPR.2011.5995344.

PERAZZI F, KRAHENBUHL P, PRITCH Y, et al. Saliency filters: Contrast based filtering for salient region detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Rhode Island, USA, 2012: 733-740. doi: 10.1109/CVPR.2012.6247743.

YANG C, ZHANG L, LU H, et al. Saliency detection via graph-based manifold ranking[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, USA, 2013: 3166-3173. doi: 10.1109/CVPR. 2013.407.

ZHU W, LIANG S, WEI Y, et al. Saliency optimization from robust background detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, USA, 2014: 2814-2821. doi: 10.1109/ CVPR. 2014.360.

QIN Y, LU H, XU Y, et al. Saliency detection via cellular automata[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, USA, 2015: 111-119. doi: 10.1109/CVPR.2015.7298606.

蒋寓文, 谭乐怡, 王守觉. 选择性背景优先的显著性检测模型[J]. 电子与信息学报, 2015, 37(1): 130-136. doi: 10.11999/JEIT140119.

JIANG Yuwen, TAN Leyi, and WANG Shoujue. Saliency detected model based on selective edges prior[J]. Journal of Electronics Information Technology, 2015, 37(1): 130-136. doi: 10.11999/JEIT140119.

吕建勇, 唐振民. 一种基于图的流形排序的显著性目标检测改进方法[J]. 电子与信息学报, 2015, 37(11): 2555-2563. doi: 10.11999/JEIT150619.

LV Jianyong and TANG Zhenmin. An improved graph-based manifold ranking for salient object detection[J]. Journal of Electronics Information Technology, 2015, 37(11): 2555-2563. doi: 10.11999/JEIT150619.

MAHADEVAN V and VASCONCELOS N. Spatiotemporal saliency in dynamic scenes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(1): 171-177. doi: 10.1109/TPAMI.2009.112.

ZHOU F, KANG S B, and COHEN M F. Time-mapping using space-time saliency[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, USA, 2014: 3358-3365. doi: 10.1109/CVPR.2014.429.

HUANG C, CHANG Y, YANG Z, et al. Video saliency map detection by dominant camera motion removal[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(8): 1336-1349. doi: 10.1109/TCSVT.2014.2308652.

LIU Z, ZHANG X, LUO S, et al. Superpixel-based spatiotemporal saliency detection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(9): 1522-1540. doi: 10.1109/TCSVT.2014.2308642.

WANG W, SHEN J, and SHAO L. Consistent video saliency using local gradient flow optimization and global refinement[J]. IEEE Transactions on Image Processing, 2015, 24(10): 1-12. doi: 10.1109/TIP.2015.2460013.

MUTHUSWAMY K and RAJAN D. Particle filter framework for salient object detection in videos[J]. IET Computer Vision, 2015, 9(3): 428-438. doi: 10.1049/ iet-cvi.2013.0298.

KIM H, KIM Y, SIM J, et al. Spatiotemporal saliency detection for video sequences based on random walk with restart[J]. IEEE Transactions on Image Processing, 2015, 24(8): 2552-2564. doi: 10.1109/TIP.2015.2425544.

WANG W, SHEN J, and PORIKLI F. Saliency-aware geodesic video object segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, USA, 2015: 3395-3402. doi: 10.1109/CVPR.2015.7298961.

KIM W and HAN J. Video saliency detection using contrast of spatiotemporal directional coherence[J]. IEEE Signal Processing Letters, 2014, 21(10): 1250-1254. doi: 10.1109/ LSP.2014.2332213.

ERDEM E and ERDEM A. Visual saliency estimation by nonlinearly integrating features using region covariances[J]. Journal of Vision, 2013, 13(4): 1-20. doi: 10.1167/13.4.11.

KOCAK A, CIZMECILERr K, ERDEM A, et al. Top down saliency estimation via superpixel-based discriminative dictionaries[C]. British Machine Vision Conference (BMVC), Nottingham, UK, 2014: 1-10. doi: 10.5244/C.28.73.

CHANG K, LIU Y, CHEN H, et al. Fusing generic objectness and visual saliency for salient object detection[C]. IEEE Conference on Computer Vision (ICCV), Barcelona, Spain, 2011: 914-921. doi: 10.1109/ICCV.2011.6126333.

LI J, TIAN Y, DUAN L, et al. Estimating visual saliency through single image optimization[J]. IEEE Signal Processing Letters, 2013, 20(9): 845-848. doi: 10.1109/LSP. 2013.2268868.

TUAEL O, PORIKLI F, and MEER P. Region covariance: A fast descriptor for detection and classification[C]. European Conference on Computer Vision, Graz, Austria, 2006: 589-600. doi: 10.1007/11744047_45.

ACHANTA R, SHAJI A, SMITH K, et al. Slic superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274-2281. doi: 10.1109/TPAMI.2012.120.

BROX T and MALIK J. Large displacement optical flow: Descriptor matching in variational motion estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(3): 500-513. doi: 10.1109/TPAMI. 2010.143.

HONG X, CHANG H, and SHAN S. Sigma set: A small second order statistical region descriptor[C]. IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, 2009: 1802-1809. doi: 10.1109/CVPR. 2009.5206742.

TSAI D, FLAGG M, and REHG J M. Motion coherent tracking with multilabel MRF optimization[C]. British Machine Vision Conference (BMVC), Aberystwyth, UK, 2010: 1-11. doi: 10.5244/C.24.56.

施引文献

资源附件(0)

访问统计