基于纹元森林和显著性先验的弱监督图像语义分割方法

韩铮; 肖志涛

doi:10.11999/JEIT170472

基于纹元森林和显著性先验的弱监督图像语义分割方法

doi: 10.11999/JEIT170472

韩铮¹,
肖志涛²

2.
(天津工业大学电子与信息工程学院天津 300387)

基金项目:

高等学校博士学科点专项科研基金(SRFDP 20131201110001)，中国纺织工业协会应用基础研究项目(J201509)，内蒙古自治区高等学校科学技术研究项目(NJZY237)

计量
- 文章访问数: 1376
- HTML全文浏览量: 145
- PDF下载量: 248
- 被引次数: 0
出版历程
- 收稿日期: 2017-05-17
- 修回日期: 2017-11-27
- 刊出日期: 2018-03-19

Weakly Supervised Semantic Segmentation Based on Semantic Texton Forest and Saliency Prior

HAN Zheng¹,
XIAO Zhitao²

2.
(School of Electronics and Information Engineering, Tianjin Polytechnic University, Tianjin 300387, China)

Funds:

The Specialized Research Fund for the Doctoral Program of Higher Education (SRFDP20131201110001), The Applied Basic Research Programs of China National Textile and Apparel Council (J201509), The Scientific Studies Program of Higher Education of Inner Mongolia Municipality (NJZY237)

摘要

摘要: 弱监督语义分割任务常利用训练集中全体图像的超像素及其相似度建立图模型，使用图像级别标记的监督关系进行约束求解。全局建模缺少单幅图像结构信息，同时此类参数方法受到复杂度限制，无法使用大规模的弱监督训练数据。针对以上问题，该文提出一种基于纹元森林和显著性先验的弱监督图像语义分割方法。算法使用弱监督数据和图像显著性训练随机森林分类器用于语义纹元森林特征(Semantic Texton Forest, STF)的提取。测试时，先将图像进行过分割，然后提取超像素语义纹元特征，利用朴素贝叶斯法进行超像素标记的概率估计，最后在条件随机场(CRF)框架下结合图像显著性信息定义了新的能量函数表达式，将图像的标注(labeling)问题转换为能量最小化问题求解。在MSRC-21类数据库上进行了验证，完成了语义分割任务。结果表明，在并未对整个训练集建立图模型的情况下，仅利用单幅图像的显著性信息也可以得到较好的分割结果，同时非参模型有利于规模数据分析。
- 语义分割 /
- 弱监督学习 /
- 显著性检测 /
- 语义纹元森林 /
- 条件随机场
Abstract: Most previous weakly supervised semantic segmentation works utilize the labels of the whole training set and thereby need the construction of a relationship graph about image labels. This method lack of structure information in single image and suffer from enormous quantity parameters which result in expensive computation. In this study, a weakly-supervised semantic segmentation algorithm is proposed. Under Conditional Random Field (CRF) framework, an novel energy function expression is developed based on saliency priors as structure context relationship, which avoids the construction of a huge graph in whole training dataset. Specifically, a nonparametric random Semantic Texton Forest (STF) is obtained using weakly supervised training data and images saliency. Then STF feature is extracted from image superpixels and probability estimates of superpixels label is calculated by naive Bayesian method. Finally, a CRF based optimization algorithm is proposed which can efficiency solved by alpha expansion algorithm. Experiments on the MSRC-21 dataset show that the new algorithm outperforms some previous influential weakly-supervised segmentation algorithms with no building graph in whole training set.
- Semantic segmentation /
- Weakly supervised learning /
- Saliency detection /
- Semantic Texton Forest (STF) /
- Conditional Random Fields (CRF)

HTML全文

参考文献(24)

KOHLI Pushmeet, LADICKY L, and TORR P H S. Robust higher order potentials for enforcing label consistency[J]. International Journal of Computer Vision, 2009, 82(3): 302-324. doi: 10.1007/s11263-008-0202-0.

ZHANG L, SONG M, LIU Z, et al. Probabilistic graphlet cut: Exploiting spatial structure cue for weakly supervised image segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 2013: 1908-1915. doi: 10.1109/CVPR.2013.249.

ZHANG Ke, ZHANG Wei, ZHENG Yingbin, et al. Sparse reconstruction for weakly supervised semantic segmentation [C]. International Joint Conference on Artificial Intelligence, Beijing, China, 2013: 1889-1895.

VEZHNEVETS A, FERRARIV, and BUHMANN J M. Weakly supervised structured output learning for semantic segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 2012: 845-852. doi: 10.1109/CVPR.2012.6247757.

VEZHNEVETS A and BUHMANN J M. Towards weakly supervised semantic segmentation by means of multiple instance and multitask learning[C]. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 2010: 3249-3256. doi: 10.1109/CVPR.2010. 5540060.

SHOTTON Jamie, JOHNSON Matthew, and CIPOLLA Roberto. Semantic texton forests for image categorization and segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA, 2008: 1-8. doi: 10.1109/CVPR.2008.4587503.

WEI Yunchao, LIANG Xiaodan, CHEN Yunpeng, et al. STC: a simple to complex framework for weakly-supervised semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(11): 2314-2320. doi: 10.1109/TPAMI.2016.2636150.

VEZHNEVETS A, FERRARI V, and BUHMANN J. M. Weakly supervised semantic segmentation with a multi-image model[C]. IEEE International Conference on Computer Vision, Washington, DC, USA, 2011: 643-650. doi: 10.1109/ ICCV.2011.6126299.

ZENG Zinan, XIAO Shijie, JIA Kui, et al. Learning by associating ambiguously labeled images[C]. IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 2013: 708-715. doi: 10.1109/CVPR.2013.97.

VEZHNEVETS A, BUHMANN J M, and FERRARI V. Active learning for semantic segmentation with expected change[C]. IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, 2012: 3162-3169. doi: 10.1109/CVPR.2012.6248050.

YING P, LIU J, and LU H. Dictionary learning based superpixels clustering for weakly-supervised semantic segmentation[C]. IEEE International Conference on Image Processing, Quebec City, QC, Canada, 2015: 4258-4262. doi: 10.1109/ICIP.2015.7351609.

OQUAB Maxime, BOTTOU Leon, LAPTEV Ivan, et al. Is object localization for free? Weakly-supervised learning with convolutional neural networks[C]. IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015: 685-694. doi: 10.1109/CVPR.2015.7298668.

PAPANDREOU George, CHEN Liang-chieh, MURPHY Kevin, et al. Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation[C]. IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 1742-1750, doi: 10.1109/ICCV.2015. 203.

PINHEIRO Pedro O and COLLOBERT Ronan. From image- level to pixel-level labeling with Convolutional Networks[C]. IEEE Conference on Computer Vision and Pattern Recognition, Boston, Massachusetts, USA, 2015: 1713-1721. doi: 10.1109/CVPR.2015.7298780.

XU Jia, SCHWING A G, and URTASUN R. Tell me what you see and i will show you where it is[C]. IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014: 3190-3197. doi: 10.1109/CVPR.2014.408.

CABRAL R, TORRE F D L, COSTEIRA J P, et al. Matrix completion for weakly-supervised multi-label image classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(1): 121-135. doi: 10.1109/ TPAMI.2014.2343234.

KOLMOGOROV Vladimir and ZABIH R. What energy functions can be minimized via graph cuts?[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(2): 147-159. doi: 10.1109/TPAMI.2004.1262177.

GEURTS Pierre, DAMIEN Ernst, and LOUIS Wehenkel. Extremely randomized trees[J]. Machine Learning, 2006, 63(1): 3-42. doi: 10.1007/s10994-006-6226-1.

JIANG Huaizu, WANG Jingdong, YUAN Zejian, et al. Salient object detection: A discriminative regional feature integration approach[J]. International Journal of Computer Vision, 2016, 9(4): 1-18. doi: 10.1007/s11263-016-0977-3.

GOFERMAN Stas, ZELNIK-MANOR Lihi, and TAL Ayellet. Context-aware saliency detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 34(10): 1915-1926. doi: 10.1109/TPAMI.2011.272.

SHOTTON Jamie, WINN John, ROTHER Carsten, et al. Texton Boost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation[C]. European Conference on Computer Vision, Graz, Austria, 2006: 1-15. doi: 10.1007/11744023-1.

LEVINSHTEIN A, STERE A, KUTULAKOS K N, et al. TurboPixels: fast superpixels using geometric flows[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(12): 2290-2297. doi: 10.1109/TPAMI.2009.96.

LADICKY L, RUSSELL C, KOHLI P, et al. Associative hierarchical random fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(6): 1056-1077. doi: 10.1109/TPAMI.2013.165.

VERBEEK J and TRIGGS B. Region classification with markov field aspect models[C]. IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 2007: 1-8. doi: 10.1109/CVPR.2007.383098.

施引文献

资源附件(0)

访问统计