可迁移测度准则下的协变量偏移修正多源集成方法

杨兴明; 吴克伟; 孙永宣; 谢昭

doi:10.11999/JEIT150323

可迁移测度准则下的协变量偏移修正多源集成方法

doi: 10.11999/JEIT150323

基金项目:

国家自然科学基金(60905005, 61273237)

计量
- 文章访问数: 1328
- HTML全文浏览量: 171
- PDF下载量: 671
- 被引次数: 0
出版历程
- 收稿日期: 2015-03-17
- 修回日期: 2015-08-13
- 刊出日期: 2015-12-19

Modified Covariate-shift Multi-source Ensemble Method in Transferability Metric

Funds:

The National Natural Science Foundation of China (60905005, 61273237)

摘要

摘要: 迁移学习通过充分利用源域共享知识，实现对目标域的小样本问题求解，然而，对训练和测试样本分布差异测度仍然是该领域的主要挑战。该文针对多源迁移学习算法中，由于源域选择和源域辅助样本选择不当引起的负迁移问题进行研究，提出一种可迁移测度准则下的协变量偏移修正多源集成方法。首先，根据源域和目标域之间的协变量偏移原则，利用联合概率的密度估计，定义辅助样本的可迁移测度，验证目标域和源域在数据空间中标记分布的一致性。其次，在多源域选择阶段，引入非迁移判别过程，提高了源域知识的迁移准确性。最后，在Caltech 256数据集中，验证了Gist特征知识表示和迁移的有效性，分析了多种条件下的辅助样本选择和源域选择的有效性。实验结果表明所提算法可有效降低负迁移现象的发生，获得更好的迁移学习性能
- 集成学习 /
- 迁移学习 /
- 协变量偏移 /
- 图像分类
Abstract: Transfer learning usually focuses on dealing with small training set in target domain by sharing knowledge generated from source ones, in which one main challenge is divergence metric of distributed samples between training and test data. In order to deal with negative transfer problem caused by improper auxiliary sample selections in source domains, this paper presents a modified covariate-shift multi-source ensemble method with transferability criterion. Firstly, transferability metric of auxiliary samples is defined by joint density estimation in accordance with co-variant transfer principles from source to target, so that the coherency of data distributions is verified. After that, whether transfer learning occurs or not should be determined after evaluating transferability metric in different sources to boost accuracy. Finally, experiments on Caltech256 using GIST demonstrate effectiveness and efficiency in the proposed approach and discussions of performance under diverse selections from auxiliary samples and source domains are presented as well. Experimental results show that the proposed method can sufficiently hold back negative transfer for better learnability in transfer style.
- Ensemble learning /
- Transfer learning /
- Covariate shift /
- Image classification

HTML全文

参考文献(32)

Tommasi T, OrabonaF, and Caputo B. Learning categories from few examples with multi model knowledge transfer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 928941.

Yao Y and Doretto G. Boosting for transfer learning with multiple sources[C]. Proceedings of Computer Vision and Pattern Recognition, San Francisco, 2010: 18551862.

Long M S, Wang J M, Ding G G, et al.. Adaptation regularization a general framework for transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(5): 10761089.

Lin D, An X, and Zhang J. Double-bootstrapping source data selection for instance-based transfer learning[J]. Pattern Recognition Letters, 2013, 34(11): 12791285.

Kuzborskij I, Orabona F, and Caputo B. From N to N+1: multiclass transfer incremental learning[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 33583365.

Zhu Y, Chen Y Q, Lu Z Q, et al.. Heterogeneous transfer learning for image classification[C]. Proceedings of AAAI Conference on Artificial Intelligence, San Francisco, 2011:　13041309.

张倩, 李明, 王雪松, 等. 一种面向多源领域的实例迁移学习[J]. 自动化学报, 2014, 40(6): 1176-1183.

Zhang Qian, Li Ming, Wang Xue-song, et al.. Instance-based transfer learning for multi-source domains[J]. Acta Automatica Sinica, 2014, 40(6): 11761183.

Pang J, Huang Q, Yan S, et al.. Transferring boosted detectors towards viewpoint and scene adaptiveness[J]. IEEE Transactions on Image Processing, 2011, 20(5): 13881400.

Li G, Qin L, Huang Q, et al.. Treat samples differently: object tracking with semi-supervised online CovBoost[C]. Proceedings of International Conference on Computer Vision, Barcelona, 2011: 627634.

Qi G J, Aggarwal C, Rui Y, et al.. Towards cross-category knowledge propagation for learning visual concepts[C]. Proceedings of Computer Vision and Pattern Recognition, Colorado Springs, 2011: 897904.

Chu W S, Torre F D, and CohnJ F. Selective transfer machine for personalized facial action unit detection[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 35153522.

Yang S Z, Hou C P, Zhang C S, et al.. Robust non-negative matrix factorization via joint sparse and graph regularization for transfer learning[J]. Neural Computing and Applications, 2013, 23(2): 541559.

方耀宁, 郭云飞, 丁雪涛, 等. 一种基于标签迁移学习的改进正则化奇异值分解推荐算法[J]. 电子与信息学报, 2013, 35(12): 30463050.

Fang Yao-ning, Guo Yun-fei, Ding Xue-tao, et al.. An improved regularized singular value decomposition recommender algorithm based on tag transfer learning[J]. Journal of Electronics Information Technology, 2013, 35(12): 30463050.

Gopalan R. Learning cross-domain information transfer for location recognition and clustering[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 731738.

Luo Y, Liu T L, Tao D C, et al.. Decomposition-based transfer distance metric learning for image classification[J]. IEEE Transactions on Image Processing, 2014, 23(9): 37893801.

Long M S, Wang J M, Ding G G, et al.. Transfer learning with graph co-regularization[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(7): 18051818.

洪佳明, 印鉴, 黄云, 等. TrSVM: 一种基于领域相似性的迁移学习算法[J]. 计算机研究与发展, 2011, 48(10): 18231830.

Hong Jia-ming, Yin Jian, Huang Yun, et al.. TrSVM: a transfer learning algorithm using domain similarity[J]. Journal of Computer Research and Development, 2011, 48(10): 18231830.

Seah C W, Tsang I W, and Ong Y S. Transfer ordinal label learning[J]. IEEE Transactions on Neural Networkand Learning System, 2013, 24(11): 18631876.

许敏, 王士同, 史荧中. 一种新的面向迁移学习的L2核分类器[J]. 电子与信息学报, 2013, 35(9): 2059-2065.

Xu Min, Wang Shi-tong, and Shi Ying-zhong. A novel transfer-learning-oriented L2 kernel classifier[J]. Journal of Electronics Information Technology, 2013, 35(9): 20592065.

Long M S, Wang J M, Ding G G, et al.. Transfer feature learning with joint distribution adaptation[C]. Proceedings of International Conference on Computer Vision, Sydney, 2013: 22002207.

Patricia N and Caputo B. Learning to learn, from transfer learning to domain adaptation: a unifying perspective[C]. Proceedings of Computer Vision and Pattern Recognition, Columbus, 2014: 14421449.

Zhang B, Wang Y, Wang Y, et al.. Stable learning in coding space for multi-class decoding and its extension for multi-class hypothesis transfer learning[C]. Proceedings of Computer Vision and Pattern Recognition, Columbus, 2014: 10751081.

Gretton A, Smola A, Huang J, et al.. Covariate Shift by Kernel Mean Matching[M]. Cambridge: MIT Press, 2009: 131160.

Huang P P, Wang G, and Qin S Y. Boosting for transfer learning from multiple data sources[J]. Pattern Recognition Letters, 2012, 33(5): 568579.

Nie Q F, Jin L Z, and Fei S M. Probability estimation for multi-class classification using AdaBoost[J]. Pattern Recognition, 2014, 47(12): 39313940.

Sugiyama M, Krauledat M, and Mller K R. Covariate shift adaptation by importance weighted cross validation[J]. The Journal of Machine Learning Research, 2007, 8(1): 9851005.

Choi M J, Lim J J, Torralba A, et al.. Exploiting hierarchical context on a large database of object categories[C]. IEEE Conference on Computer Vision Pattern Recognition, San Frencisco, CA, 2010: 129136.

Han Y and Liu G. Biologically inspired task oriented gist model for scene classification[J]. Computer Vision and Image Understanding, 2013, 117(1): 7695.

施引文献

资源附件(0)

访问统计