A Local Spatiotemporal Optimization Framework for Video Saliency Detection Using Region Covariance

TIAN Chang; JIANG Qingzhu; WU Zemin; LIU Tao; HU Lei

doi:10.11999/JEIT151122

Volume 38 Issue 7

Jul. 2016

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2016 > 38(7): 1586-1593

TIAN Chang, JIANG Qingzhu, WU Zemin, LIU Tao, HU Lei. A Local Spatiotemporal Optimization Framework for Video Saliency Detection Using Region Covariance[J]. Journal of Electronics & Information Technology, 2016, 38(7): 1586-1593. doi: 10.11999/JEIT151122

Citation:

TIAN Chang, JIANG Qingzhu, WU Zemin, LIU Tao, HU Lei. A Local Spatiotemporal Optimization Framework for Video Saliency Detection Using Region Covariance[J]. Journal of Electronics & Information Technology, 2016, 38(7): 1586-1593. doi: 10.11999/JEIT151122

Citation:

PDF( 6760 KB)

A Local Spatiotemporal Optimization Framework for Video Saliency Detection Using Region Covariance

doi: 10.11999/JEIT151122 cstr: 32379.14.JEIT151122

Funds:

The National Natural Science Youth Foundation of China (61501509)

Received Date: 2015-10-08
Rev Recd Date: 2016-02-29
Publish Date: 2016-07-19

Abstract

Abstract

Visual saliency is widely applied to computer vision. Image saliency detection has been extensively studied, while there are only a few effective methods of computing saliency for videos owing to its high challenge. Inspired by image saliency methods, this paper proposes a unified spatiotemporal feature extraction and optimization framework for video saliency. First, the spatiotemporal feature descriptor is constructed via region covariance. Then, initial saliency map is computed by the local contrast of the descriptor. Finally, a local spatiotemporal optimization framework considering the previous and next frames of the current one is modeled to obtain the final saliency map. Extensive experiments on two public datasets demonstrate that the proposed algorithm not only outperforms the state-of-the-art methods, but also is of great extendibility.
- Video saliency,
- Region covariance,
- Local contrast,
- Local spatiotemporal optimization

FullText(HTML)

References(33)

References

BORJI A, CHENG M, JIANG H, et al. Salient object detection: A benchmark[J]. IEEE Transactions on Image Processing, 2015, 24(12): 5706-5722. doi: 10.1109/ TIP.2015.2487833.

ROTHER C, KOLMOGOROV V, and BLAKE A. Grabcut: Interactive foreground extraction using iterated graph cuts[J]. ACM Transactions on Graphics, 2004, 23(1): 309-314. doi: 10.1145/1186562.1015720.

DING Y, XIAO J, and YU J. Importance filtering for image retargeting[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, 2011: 89-96. doi: 10.1109/CVPR.2011.5995445.

MAHADEVAN V and VASOONCEIOS N. Saliency-based discriminant tracking[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, USA, 2009: 1007-1013. doi: 10.1109/CVPR.2009.5206573.

SHARMA G, JURIE F, and SCHMID C. Discriminative spatial saliency for image classification[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Rhode Island, USA, 2012: 3506-3513. doi: 10.1109/ CVPR.2012.6248093.

HADIZADEH H and BAJIC I. Saliency-aware video compression[J]. IEEE Transactions on Image Processing, 2014, 23(1): 19-33. doi: 10.1109/TIP.2013.2282897.

CHENG M, ZHANG G, MIERA N, et al. Global contrast based salient region detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, 2011: 409-416. doi: 10.1109/ CVPR.2011.5995344.

PERAZZI F, KRAHENBUHL P, PRITCH Y, et al. Saliency filters: Contrast based filtering for salient region detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Rhode Island, USA, 2012: 733-740. doi: 10.1109/CVPR.2012.6247743.

YANG C, ZHANG L, LU H, et al. Saliency detection via graph-based manifold ranking[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, USA, 2013: 3166-3173. doi: 10.1109/CVPR. 2013.407.

ZHU W, LIANG S, WEI Y, et al. Saliency optimization from robust background detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, USA, 2014: 2814-2821. doi: 10.1109/ CVPR. 2014.360.

QIN Y, LU H, XU Y, et al. Saliency detection via cellular automata[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, USA, 2015: 111-119. doi: 10.1109/CVPR.2015.7298606.

蒋寓文, 谭乐怡, 王守觉. 选择性背景优先的显著性检测模型[J]. 电子与信息学报, 2015, 37(1): 130-136. doi: 10.11999/JEIT140119.

JIANG Yuwen, TAN Leyi, and WANG Shoujue. Saliency detected model based on selective edges prior[J]. Journal of Electronics Information Technology, 2015, 37(1): 130-136. doi: 10.11999/JEIT140119.

吕建勇, 唐振民. 一种基于图的流形排序的显著性目标检测改进方法[J]. 电子与信息学报, 2015, 37(11): 2555-2563. doi: 10.11999/JEIT150619.

LV Jianyong and TANG Zhenmin. An improved graph-based manifold ranking for salient object detection[J]. Journal of Electronics Information Technology, 2015, 37(11): 2555-2563. doi: 10.11999/JEIT150619.

MAHADEVAN V and VASCONCELOS N. Spatiotemporal saliency in dynamic scenes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(1): 171-177. doi: 10.1109/TPAMI.2009.112.

ZHOU F, KANG S B, and COHEN M F. Time-mapping using space-time saliency[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, USA, 2014: 3358-3365. doi: 10.1109/CVPR.2014.429.

HUANG C, CHANG Y, YANG Z, et al. Video saliency map detection by dominant camera motion removal[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(8): 1336-1349. doi: 10.1109/TCSVT.2014.2308652.

LIU Z, ZHANG X, LUO S, et al. Superpixel-based spatiotemporal saliency detection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(9): 1522-1540. doi: 10.1109/TCSVT.2014.2308642.

WANG W, SHEN J, and SHAO L. Consistent video saliency using local gradient flow optimization and global refinement[J]. IEEE Transactions on Image Processing, 2015, 24(10): 1-12. doi: 10.1109/TIP.2015.2460013.

MUTHUSWAMY K and RAJAN D. Particle filter framework for salient object detection in videos[J]. IET Computer Vision, 2015, 9(3): 428-438. doi: 10.1049/ iet-cvi.2013.0298.

KIM H, KIM Y, SIM J, et al. Spatiotemporal saliency detection for video sequences based on random walk with restart[J]. IEEE Transactions on Image Processing, 2015, 24(8): 2552-2564. doi: 10.1109/TIP.2015.2425544.

WANG W, SHEN J, and PORIKLI F. Saliency-aware geodesic video object segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, USA, 2015: 3395-3402. doi: 10.1109/CVPR.2015.7298961.

KIM W and HAN J. Video saliency detection using contrast of spatiotemporal directional coherence[J]. IEEE Signal Processing Letters, 2014, 21(10): 1250-1254. doi: 10.1109/ LSP.2014.2332213.

ERDEM E and ERDEM A. Visual saliency estimation by nonlinearly integrating features using region covariances[J]. Journal of Vision, 2013, 13(4): 1-20. doi: 10.1167/13.4.11.

KOCAK A, CIZMECILERr K, ERDEM A, et al. Top down saliency estimation via superpixel-based discriminative dictionaries[C]. British Machine Vision Conference (BMVC), Nottingham, UK, 2014: 1-10. doi: 10.5244/C.28.73.

CHANG K, LIU Y, CHEN H, et al. Fusing generic objectness and visual saliency for salient object detection[C]. IEEE Conference on Computer Vision (ICCV), Barcelona, Spain, 2011: 914-921. doi: 10.1109/ICCV.2011.6126333.

LI J, TIAN Y, DUAN L, et al. Estimating visual saliency through single image optimization[J]. IEEE Signal Processing Letters, 2013, 20(9): 845-848. doi: 10.1109/LSP. 2013.2268868.

TUAEL O, PORIKLI F, and MEER P. Region covariance: A fast descriptor for detection and classification[C]. European Conference on Computer Vision, Graz, Austria, 2006: 589-600. doi: 10.1007/11744047_45.

ACHANTA R, SHAJI A, SMITH K, et al. Slic superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274-2281. doi: 10.1109/TPAMI.2012.120.

BROX T and MALIK J. Large displacement optical flow: Descriptor matching in variational motion estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(3): 500-513. doi: 10.1109/TPAMI. 2010.143.

HONG X, CHANG H, and SHAN S. Sigma set: A small second order statistical region descriptor[C]. IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, 2009: 1802-1809. doi: 10.1109/CVPR. 2009.5206742.

TSAI D, FLAGG M, and REHG J M. Motion coherent tracking with multilabel MRF optimization[C]. British Machine Vision Conference (BMVC), Aberystwyth, UK, 2010: 1-11. doi: 10.5244/C.24.56.

Relative Articles

Supplements(0)

Cited By

Proportional views