Modified Covariate-shift Multi-source Ensemble Method in Transferability Metric

Yang Xing-ming; Wu Ke-wei; Sun Yong-xuan; Xie Zhao

doi:10.11999/JEIT150323

Volume 37 Issue 12

Jan. 2016

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2015 > 37(12): 2913-2920

Yang Xing-ming, Wu Ke-wei, Sun Yong-xuan, Xie Zhao. Modified Covariate-shift Multi-source Ensemble Method in Transferability Metric[J]. Journal of Electronics & Information Technology, 2015, 37(12): 2913-2920. doi: 10.11999/JEIT150323

Citation:

Yang Xing-ming, Wu Ke-wei, Sun Yong-xuan, Xie Zhao. Modified Covariate-shift Multi-source Ensemble Method in Transferability Metric[J]. Journal of Electronics & Information Technology, 2015, 37(12): 2913-2920. doi: 10.11999/JEIT150323

Citation:

PDF( 652 KB)

Modified Covariate-shift Multi-source Ensemble Method in Transferability Metric

doi: 10.11999/JEIT150323

Funds:

The National Natural Science Foundation of China (60905005, 61273237)

Received Date: 2015-03-17
Rev Recd Date: 2015-08-13
Publish Date: 2015-12-19

Abstract

Abstract

Transfer learning usually focuses on dealing with small training set in target domain by sharing knowledge generated from source ones, in which one main challenge is divergence metric of distributed samples between training and test data. In order to deal with negative transfer problem caused by improper auxiliary sample selections in source domains, this paper presents a modified covariate-shift multi-source ensemble method with transferability criterion. Firstly, transferability metric of auxiliary samples is defined by joint density estimation in accordance with co-variant transfer principles from source to target, so that the coherency of data distributions is verified. After that, whether transfer learning occurs or not should be determined after evaluating transferability metric in different sources to boost accuracy. Finally, experiments on Caltech256 using GIST demonstrate effectiveness and efficiency in the proposed approach and discussions of performance under diverse selections from auxiliary samples and source domains are presented as well. Experimental results show that the proposed method can sufficiently hold back negative transfer for better learnability in transfer style.
- Ensemble learning,
- Transfer learning,
- Covariate shift,
- Image classification

FullText(HTML)

References(32)

References

Tommasi T, OrabonaF, and Caputo B. Learning categories from few examples with multi model knowledge transfer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 928941.

Yao Y and Doretto G. Boosting for transfer learning with multiple sources[C]. Proceedings of Computer Vision and Pattern Recognition, San Francisco, 2010: 18551862.

Long M S, Wang J M, Ding G G, et al.. Adaptation regularization a general framework for transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(5): 10761089.

Lin D, An X, and Zhang J. Double-bootstrapping source data selection for instance-based transfer learning[J]. Pattern Recognition Letters, 2013, 34(11): 12791285.

Kuzborskij I, Orabona F, and Caputo B. From N to N+1: multiclass transfer incremental learning[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 33583365.

Zhu Y, Chen Y Q, Lu Z Q, et al.. Heterogeneous transfer learning for image classification[C]. Proceedings of AAAI Conference on Artificial Intelligence, San Francisco, 2011:　13041309.

张倩, 李明, 王雪松, 等. 一种面向多源领域的实例迁移学习[J]. 自动化学报, 2014, 40(6): 1176-1183.

Zhang Qian, Li Ming, Wang Xue-song, et al.. Instance-based transfer learning for multi-source domains[J]. Acta Automatica Sinica, 2014, 40(6): 11761183.

Pang J, Huang Q, Yan S, et al.. Transferring boosted detectors towards viewpoint and scene adaptiveness[J]. IEEE Transactions on Image Processing, 2011, 20(5): 13881400.

Li G, Qin L, Huang Q, et al.. Treat samples differently: object tracking with semi-supervised online CovBoost[C]. Proceedings of International Conference on Computer Vision, Barcelona, 2011: 627634.

Qi G J, Aggarwal C, Rui Y, et al.. Towards cross-category knowledge propagation for learning visual concepts[C]. Proceedings of Computer Vision and Pattern Recognition, Colorado Springs, 2011: 897904.

Chu W S, Torre F D, and CohnJ F. Selective transfer machine for personalized facial action unit detection[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 35153522.

Yang S Z, Hou C P, Zhang C S, et al.. Robust non-negative matrix factorization via joint sparse and graph regularization for transfer learning[J]. Neural Computing and Applications, 2013, 23(2): 541559.

方耀宁, 郭云飞, 丁雪涛, 等. 一种基于标签迁移学习的改进正则化奇异值分解推荐算法[J]. 电子与信息学报, 2013, 35(12): 30463050.

Fang Yao-ning, Guo Yun-fei, Ding Xue-tao, et al.. An improved regularized singular value decomposition recommender algorithm based on tag transfer learning[J]. Journal of Electronics Information Technology, 2013, 35(12): 30463050.

Gopalan R. Learning cross-domain information transfer for location recognition and clustering[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 731738.

Luo Y, Liu T L, Tao D C, et al.. Decomposition-based transfer distance metric learning for image classification[J]. IEEE Transactions on Image Processing, 2014, 23(9): 37893801.

Long M S, Wang J M, Ding G G, et al.. Transfer learning with graph co-regularization[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(7): 18051818.

洪佳明, 印鉴, 黄云, 等. TrSVM: 一种基于领域相似性的迁移学习算法[J]. 计算机研究与发展, 2011, 48(10): 18231830.

Hong Jia-ming, Yin Jian, Huang Yun, et al.. TrSVM: a transfer learning algorithm using domain similarity[J]. Journal of Computer Research and Development, 2011, 48(10): 18231830.

Seah C W, Tsang I W, and Ong Y S. Transfer ordinal label learning[J]. IEEE Transactions on Neural Networkand Learning System, 2013, 24(11): 18631876.

许敏, 王士同, 史荧中. 一种新的面向迁移学习的L2核分类器[J]. 电子与信息学报, 2013, 35(9): 2059-2065.

Xu Min, Wang Shi-tong, and Shi Ying-zhong. A novel transfer-learning-oriented L2 kernel classifier[J]. Journal of Electronics Information Technology, 2013, 35(9): 20592065.

Long M S, Wang J M, Ding G G, et al.. Transfer feature learning with joint distribution adaptation[C]. Proceedings of International Conference on Computer Vision, Sydney, 2013: 22002207.

Patricia N and Caputo B. Learning to learn, from transfer learning to domain adaptation: a unifying perspective[C]. Proceedings of Computer Vision and Pattern Recognition, Columbus, 2014: 14421449.

Zhang B, Wang Y, Wang Y, et al.. Stable learning in coding space for multi-class decoding and its extension for multi-class hypothesis transfer learning[C]. Proceedings of Computer Vision and Pattern Recognition, Columbus, 2014: 10751081.

Gretton A, Smola A, Huang J, et al.. Covariate Shift by Kernel Mean Matching[M]. Cambridge: MIT Press, 2009: 131160.

Huang P P, Wang G, and Qin S Y. Boosting for transfer learning from multiple data sources[J]. Pattern Recognition Letters, 2012, 33(5): 568579.

Nie Q F, Jin L Z, and Fei S M. Probability estimation for multi-class classification using AdaBoost[J]. Pattern Recognition, 2014, 47(12): 39313940.

Sugiyama M, Krauledat M, and Mller K R. Covariate shift adaptation by importance weighted cross validation[J]. The Journal of Machine Learning Research, 2007, 8(1): 9851005.

Choi M J, Lim J J, Torralba A, et al.. Exploiting hierarchical context on a large database of object categories[C]. IEEE Conference on Computer Vision Pattern Recognition, San Frencisco, CA, 2010: 129136.

Han Y and Liu G. Biologically inspired task oriented gist model for scene classification[J]. Computer Vision and Image Understanding, 2013, 117(1): 7695.

Relative Articles

Supplements(0)

Cited By

Proportional views