Weakly Supervised Semantic Segmentation Based on Semantic Texton Forest and Saliency Prior

HAN Zheng; XIAO Zhitao

doi:10.11999/JEIT170472

Volume 40 Issue 3

Mar. 2018

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2018 > 40(3): 610-617

HAN Zheng, XIAO Zhitao. Weakly Supervised Semantic Segmentation Based on Semantic Texton Forest and Saliency Prior[J]. Journal of Electronics & Information Technology, 2018, 40(3): 610-617. doi: 10.11999/JEIT170472

Citation:

HAN Zheng, XIAO Zhitao. Weakly Supervised Semantic Segmentation Based on Semantic Texton Forest and Saliency Prior[J]. Journal of Electronics & Information Technology, 2018, 40(3): 610-617. doi: 10.11999/JEIT170472

Citation:

PDF( 16755 KB)

Weakly Supervised Semantic Segmentation Based on Semantic Texton Forest and Saliency Prior

doi: 10.11999/JEIT170472

HAN Zheng¹,
XIAO Zhitao²

1.
2.
(School of Electronics and Information Engineering, Tianjin Polytechnic University, Tianjin 300387, China)

Funds:

The Specialized Research Fund for the Doctoral Program of Higher Education (SRFDP20131201110001), The Applied Basic Research Programs of China National Textile and Apparel Council (J201509), The Scientific Studies Program of Higher Education of Inner Mongolia Municipality (NJZY237)

Received Date: 2017-05-17
Rev Recd Date: 2017-11-27
Publish Date: 2018-03-19

Abstract

Abstract

Most previous weakly supervised semantic segmentation works utilize the labels of the whole training set and thereby need the construction of a relationship graph about image labels. This method lack of structure information in single image and suffer from enormous quantity parameters which result in expensive computation. In this study, a weakly-supervised semantic segmentation algorithm is proposed. Under Conditional Random Field (CRF) framework, an novel energy function expression is developed based on saliency priors as structure context relationship, which avoids the construction of a huge graph in whole training dataset. Specifically, a nonparametric random Semantic Texton Forest (STF) is obtained using weakly supervised training data and images saliency. Then STF feature is extracted from image superpixels and probability estimates of superpixels label is calculated by naive Bayesian method. Finally, a CRF based optimization algorithm is proposed which can efficiency solved by alpha expansion algorithm. Experiments on the MSRC-21 dataset show that the new algorithm outperforms some previous influential weakly-supervised segmentation algorithms with no building graph in whole training set.
- Semantic segmentation,
- Weakly supervised learning,
- Saliency detection,
- Semantic Texton Forest (STF),
- Conditional Random Fields (CRF)

FullText(HTML)

References(24)

References

KOHLI Pushmeet, LADICKY L, and TORR P H S. Robust higher order potentials for enforcing label consistency[J]. International Journal of Computer Vision, 2009, 82(3): 302-324. doi: 10.1007/s11263-008-0202-0.

ZHANG L, SONG M, LIU Z, et al. Probabilistic graphlet cut: Exploiting spatial structure cue for weakly supervised image segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 2013: 1908-1915. doi: 10.1109/CVPR.2013.249.

ZHANG Ke, ZHANG Wei, ZHENG Yingbin, et al. Sparse reconstruction for weakly supervised semantic segmentation [C]. International Joint Conference on Artificial Intelligence, Beijing, China, 2013: 1889-1895.

VEZHNEVETS A, FERRARIV, and BUHMANN J M. Weakly supervised structured output learning for semantic segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 2012: 845-852. doi: 10.1109/CVPR.2012.6247757.

VEZHNEVETS A and BUHMANN J M. Towards weakly supervised semantic segmentation by means of multiple instance and multitask learning[C]. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 2010: 3249-3256. doi: 10.1109/CVPR.2010. 5540060.

SHOTTON Jamie, JOHNSON Matthew, and CIPOLLA Roberto. Semantic texton forests for image categorization and segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA, 2008: 1-8. doi: 10.1109/CVPR.2008.4587503.

WEI Yunchao, LIANG Xiaodan, CHEN Yunpeng, et al. STC: a simple to complex framework for weakly-supervised semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(11): 2314-2320. doi: 10.1109/TPAMI.2016.2636150.

VEZHNEVETS A, FERRARI V, and BUHMANN J. M. Weakly supervised semantic segmentation with a multi-image model[C]. IEEE International Conference on Computer Vision, Washington, DC, USA, 2011: 643-650. doi: 10.1109/ ICCV.2011.6126299.

ZENG Zinan, XIAO Shijie, JIA Kui, et al. Learning by associating ambiguously labeled images[C]. IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 2013: 708-715. doi: 10.1109/CVPR.2013.97.

VEZHNEVETS A, BUHMANN J M, and FERRARI V. Active learning for semantic segmentation with expected change[C]. IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, 2012: 3162-3169. doi: 10.1109/CVPR.2012.6248050.

YING P, LIU J, and LU H. Dictionary learning based superpixels clustering for weakly-supervised semantic segmentation[C]. IEEE International Conference on Image Processing, Quebec City, QC, Canada, 2015: 4258-4262. doi: 10.1109/ICIP.2015.7351609.

OQUAB Maxime, BOTTOU Leon, LAPTEV Ivan, et al. Is object localization for free? Weakly-supervised learning with convolutional neural networks[C]. IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015: 685-694. doi: 10.1109/CVPR.2015.7298668.

PAPANDREOU George, CHEN Liang-chieh, MURPHY Kevin, et al. Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation[C]. IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 1742-1750, doi: 10.1109/ICCV.2015. 203.

PINHEIRO Pedro O and COLLOBERT Ronan. From image- level to pixel-level labeling with Convolutional Networks[C]. IEEE Conference on Computer Vision and Pattern Recognition, Boston, Massachusetts, USA, 2015: 1713-1721. doi: 10.1109/CVPR.2015.7298780.

XU Jia, SCHWING A G, and URTASUN R. Tell me what you see and i will show you where it is[C]. IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014: 3190-3197. doi: 10.1109/CVPR.2014.408.

CABRAL R, TORRE F D L, COSTEIRA J P, et al. Matrix completion for weakly-supervised multi-label image classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(1): 121-135. doi: 10.1109/ TPAMI.2014.2343234.

KOLMOGOROV Vladimir and ZABIH R. What energy functions can be minimized via graph cuts?[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(2): 147-159. doi: 10.1109/TPAMI.2004.1262177.

GEURTS Pierre, DAMIEN Ernst, and LOUIS Wehenkel. Extremely randomized trees[J]. Machine Learning, 2006, 63(1): 3-42. doi: 10.1007/s10994-006-6226-1.

JIANG Huaizu, WANG Jingdong, YUAN Zejian, et al. Salient object detection: A discriminative regional feature integration approach[J]. International Journal of Computer Vision, 2016, 9(4): 1-18. doi: 10.1007/s11263-016-0977-3.

GOFERMAN Stas, ZELNIK-MANOR Lihi, and TAL Ayellet. Context-aware saliency detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 34(10): 1915-1926. doi: 10.1109/TPAMI.2011.272.

SHOTTON Jamie, WINN John, ROTHER Carsten, et al. Texton Boost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation[C]. European Conference on Computer Vision, Graz, Austria, 2006: 1-15. doi: 10.1007/11744023-1.

LEVINSHTEIN A, STERE A, KUTULAKOS K N, et al. TurboPixels: fast superpixels using geometric flows[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(12): 2290-2297. doi: 10.1109/TPAMI.2009.96.

LADICKY L, RUSSELL C, KOHLI P, et al. Associative hierarchical random fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(6): 1056-1077. doi: 10.1109/TPAMI.2013.165.

VERBEEK J and TRIGGS B. Region classification with markov field aspect models[C]. IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 2007: 1-8. doi: 10.1109/CVPR.2007.383098.

Relative Articles

Supplements(0)

Cited By

Proportional views