最小化类内距离和分类算法

王晓初; 王士同; 包芳; 蒋亦樟

doi:10.11999/JEIT150633

最小化类内距离和分类算法

doi: 10.11999/JEIT150633

2.
(江南大学数字媒体学院无锡 214122) ②(江苏省信息融合软件工程技术研发中心江阴 214405)

基金项目:

国家自然科学基金(61170122, 61272210)

计量
- 文章访问数: 1342
- HTML全文浏览量: 212
- PDF下载量: 793
- 被引次数: 0
出版历程
- 收稿日期: 2015-05-27
- 修回日期: 2015-09-22
- 刊出日期: 2016-03-19

Intraclass-Distance-Sum-Minimization Based Classification Algorithm

2.
(School of Digital Media, Jiangnan University, Wuxi 214122, China)

Funds:

The National Natural Science Foundation of China (61170122, 61272210)

摘要

摘要: 支持向量机分类算法引入惩罚因子来调节过拟合和线性不可分时无解的问题，优点是可以通过调节参数取得最优解，但带来的问题是允许一部分样本错分。错分的样本在分类间隔之间失去了约束，导致两类交界处样本杂乱分布，并且增加了训练的负担。为了解决上述问题，该文根据大间隔分类思想，基于类内紧密类间松散的原则，提出一种新的分类算法，称之为最小化类内距离和(Intraclass-Distance-Sum-Minimization, IDSM)分类算法。该算法根据最小化类内距离和准则构造训练模型，通过解析法求解得到最佳的映射法则，进而利用该最佳映射法则对样本进行投影变换以达到类内间隔小类间间隔大的效果。相应地，为解决高维样本分类问题，进一步提出了该文算法的核化版本。在大量UCI数据集和Yale大学人脸数据库上的实验结果表明了该文算法的优越性。
- 支持向量机 /
- 惩罚因子 /
- 大间隔分类思想 /
- 类内距离和 /
- 映射法则
Abstract: Classification algorithm of Support Vector Machine (SVM) is introduced the penalty factor to adjust the overfit and nonlinear problem. The method is beneficial for seeking the optimal solution by allowing a part of samples error classified. But it also causes a problem that error classified samples distribute disorderedly and increase the burden of training. In order to solve the above problems, according to large margin classification thought, based on principles that the intraclass samples must be closer and the interclass samples must be looser, this research proposes a new classification algorithm called Intraclass-Distance-Sum-Minimization (IDSM) based classification algorithm. This algorithm constructs a training model by using principle of minimizing the sum of the intraclass distance and finds the optimal projection rule by analytical method. And then the optimal projection rule is used to samples projection transformation to achieve the effect that intraclass intervals are small and the interclass intervals are large. Accordingly, this research offers a kernel version of the algorithm to solve high-dimensional classification problems. Experiment results on a large number of UCI datasets and the Yale face database indicate the superiority of the proposed algorithm.
- Support Vector Machine (SVM) /
- Penalty factor /
- Large margin classification thought /
- Sum of intraclass distance /
- Projection rule

HTML全文

参考文献(38)

QUINLAN J R. Induction of decision trees[J]. Machine Learning, 1986, 1(1): 81-106.

QUINLAN J R. Improved use of continuous attributes in C4.5[J]. Journal of Artificial Intelligence Research, 1996, 4(1): 77-90.

PENG F, SCHUURMANS D, and WANG S. Augmenting naive Bayes classifiers with statistical language models[J]. Information Retrieval, 2004, 7(3/4): 317-345.

CHENG J and GREINER R. Comparing Bayesian network classifiers[C]. Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, San Francisco, USA, 1999: 101-108.

COVER T and HART P. Nearest neighbor pattern classification [J]. IEEE Transactions on Information Theory, 1967, 13(1): 21-27.

BIJALWAN V, KUMAR V, KUMARI P, et al. KNN based machine learning approach for text and document mining[J]. International Journal of Database Theory and Application, 2014, 7(1): 61-70.

黄剑华, 丁建睿, 刘家锋, 等. 基于局部加权的Citation-kNN算法[J]. 电子与信息学报, 2013, 35(3): 627-632.

HUANG Jianhua, DING Jianrui, LIU Jiafeng, et al. Citation- kNN algorithm based on locally-weighting[J]. Journal of Electronics Information Technology, 2013, 35(3): 627-632.

WELLING M. Fisher linear discriminant analysis[J]. Department of Computer Science, 2008, 16(94): 237-280.

FUIN N, PEDEMONTE S, ARRIDGE S, et al. Efficient determination of the uncertainty for the optimization of SPECT system design: a subsampled fisher information matrix[J]. IEEE Transactions on Medical Imaging, 2014, 33(3): 618-635.

DUFRENOIS F. A one-class kernel fisher criterion for outlier detection[J]. IEEE Transactions on Neural Networks Learning Systems, 2014, 26(5): 982-994.

VAN Ooyen A and NIENHUIS B. Improving the convergence of the back-propagation algorithm[J]. Neural Networks, 1992, 5(3): 465-471.

潘舟浩, 李道京, 刘波, 等. 基于BP算法和时变基线的机载InSAR数据处理方法研究[J]. 电子与信息学报, 2014, 36(7): 1585-1591.

PAN Zhouhao, LI Daojing, LIU Bo, et al. Processing of the airborne InSAR data based on the BP algorithm and the time-varying baseline[J] Journal of Electronics Information Technology, 2014, 36(7): 1585-1591.

SUYKENS J A K and VANDEWALLE J. Least squares support vector machine classifiers[J]. Neural Processing Letters, 1999, 9(3): 293-300.

胡文军, 王士同, 王娟, 等. 非线性分类的分割超平面快速集成方法[J]. 电子与信息学报, 2012, 34(3): 535-542.

HU Wenjun, WANG Shitong, WANG Juan, et al. Fast ensemble of separating hyperplanes for nonlinear classification[J]. Journal of Electronics Information Technology, 2012, 34(3): 535-542.

GAO X, LU T, LIU P, et al. A soil moisture classification model based on SVM used in agricultural WSN[C]. 2014 IEEE 7th Joint International, Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing, 2014: 432-436.

RIES C X, RICHTER F, ROMBERG S, et al. Automatic object annotation from weakly labeled data with latent structured SVM[C]. 2014 12th IEEE International Workshop on Content-based Multimedia Indexing (CBMI), Klagenfurt, Austria, 2014: 1-4.

PLATT J. Fast training of support vector machines using sequential minimal optimization[J]. Advances in Kernel Methods: Support Vector Learning, 1999(3): 185-208.

JOACHIMS T. Making large scale SVM learning practical[R]. Universit?t Dortmund, 1999.

CHANG C C and LIN C J. LIBSVM: A library for support vector machines[J]. ACM Transactions on Intelligent Systems Technology, 2011, 2(3): 389-396.

MANGASARIAN O L and MUSICANT D R. Lagrangian support vector machines[J]. The Journal of Machine Learning Research, 2001, 1(3): 161-177.

SEOK K. Semi-supervised regression based on support vector machine[J]. Computer Engineering Applications, 2014, 25(2): 447-454.

LENG Y, XU X, and QI G. Combining active learning and semi-supervised learning to construct SVM classifier[J]. Knowledge-Based Systems, 2013, 44(1): 121-131.

CHEN W J, SHAO Y H, XU D K, et al. Manifold proximal support vector machine for semi-supervised classification[J]. Applied Intelligence, 2014, 40(4): 623-638.

李红莲, 王春花, 袁保宗. 一种改进的支持向量机NN- SVM[J]. 计算机学报, 2003, 26(8): 1015-1020.

LI Honglian, WANG Chunhua, and YUAN Baozong. An improved SVM: NN-SVM[J]. Chinese Journal of Computers, 2003, 26(8): 1015-1020.

陈宝林. 最优化理论与算法[M]. 北京: 清华大学出版社, 2005: 281-322.

CHEN Baolin. Optimization Theory and Algorithm[M]. Beijing, Tsinghua University Press, 2005: 281-322.

YOSHIYAMA K and SAKURAI A. Laplacian minimax probability machine[J]. Pattern Recognition Letters, 2014, 37: 192-200.

MIGLIORATI G. Adaptive polynomial approximation by means of random discrete least squares[J]. Lecture Notes in Computational Science Engineering, 2013, 103: 547-554.

HUANG K, YANG H, KING I, et al. The minimum error minimax probability machine[J]. The Journal of Machine Learning Research, 2004(5): 1253-1286.

PLAN Y and VERSHYNIN R. Robust 1-bit compressed sensing and sparse logistic regression: A convex programming approach[J]. IEEE Transactions on Information Theory, 2013, 59(1): 482-494.

ALIZADEN F. Interior point methods in semidefinite programming with applications to combinatorial optimization[J]. SIAM Journal on Optimization, 1995, 5(1): 13-51.

BOYD S and VANDENBERGHE L. Convex Optimi- zation[M]. Cambridge University Press, 2009: 127- 214.

边肇祺, 张学工. 模式识别[M]. 北京: 清华大学出版社, 2001: 83-90.

BIAN Zhaoqi and ZHANG Xuegong. Pattern Recognition[M]. Beijing, Tsinghua University Press, 2001: 83-90.

施引文献

资源附件(0)

访问统计