Intraclass-Distance-Sum-Minimization Based Classification Algorithm

WANG Xiaochu; WANG Shitong; BAO Fang; JIANG Yizhang

doi:10.11999/JEIT150633

Volume 38 Issue 3

Mar. 2016

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2016 > 38(3): 532-540

WANG Xiaochu, WANG Shitong, BAO Fang, JIANG Yizhang. Intraclass-Distance-Sum-Minimization Based Classification Algorithm[J]. Journal of Electronics & Information Technology, 2016, 38(3): 532-540. doi: 10.11999/JEIT150633

Citation:

WANG Xiaochu, WANG Shitong, BAO Fang, JIANG Yizhang. Intraclass-Distance-Sum-Minimization Based Classification Algorithm[J]. Journal of Electronics & Information Technology, 2016, 38(3): 532-540. doi: 10.11999/JEIT150633

Citation:

PDF( 690 KB)

Intraclass-Distance-Sum-Minimization Based Classification Algorithm

doi: 10.11999/JEIT150633

1.
2.
(School of Digital Media, Jiangnan University, Wuxi 214122, China)

Funds:

The National Natural Science Foundation of China (61170122, 61272210)

Received Date: 2015-05-27
Rev Recd Date: 2015-09-22
Publish Date: 2016-03-19

Abstract

Abstract

Classification algorithm of Support Vector Machine (SVM) is introduced the penalty factor to adjust the overfit and nonlinear problem. The method is beneficial for seeking the optimal solution by allowing a part of samples error classified. But it also causes a problem that error classified samples distribute disorderedly and increase the burden of training. In order to solve the above problems, according to large margin classification thought, based on principles that the intraclass samples must be closer and the interclass samples must be looser, this research proposes a new classification algorithm called Intraclass-Distance-Sum-Minimization (IDSM) based classification algorithm. This algorithm constructs a training model by using principle of minimizing the sum of the intraclass distance and finds the optimal projection rule by analytical method. And then the optimal projection rule is used to samples projection transformation to achieve the effect that intraclass intervals are small and the interclass intervals are large. Accordingly, this research offers a kernel version of the algorithm to solve high-dimensional classification problems. Experiment results on a large number of UCI datasets and the Yale face database indicate the superiority of the proposed algorithm.
- Support Vector Machine (SVM),
- Penalty factor,
- Large margin classification thought,
- Sum of intraclass distance,
- Projection rule

FullText(HTML)

References(38)

References

QUINLAN J R. Induction of decision trees[J]. Machine Learning, 1986, 1(1): 81-106.

QUINLAN J R. Improved use of continuous attributes in C4.5[J]. Journal of Artificial Intelligence Research, 1996, 4(1): 77-90.

PENG F, SCHUURMANS D, and WANG S. Augmenting naive Bayes classifiers with statistical language models[J]. Information Retrieval, 2004, 7(3/4): 317-345.

CHENG J and GREINER R. Comparing Bayesian network classifiers[C]. Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, San Francisco, USA, 1999: 101-108.

COVER T and HART P. Nearest neighbor pattern classification [J]. IEEE Transactions on Information Theory, 1967, 13(1): 21-27.

BIJALWAN V, KUMAR V, KUMARI P, et al. KNN based machine learning approach for text and document mining[J]. International Journal of Database Theory and Application, 2014, 7(1): 61-70.

黄剑华, 丁建睿, 刘家锋, 等. 基于局部加权的Citation-kNN算法[J]. 电子与信息学报, 2013, 35(3): 627-632.

HUANG Jianhua, DING Jianrui, LIU Jiafeng, et al. Citation- kNN algorithm based on locally-weighting[J]. Journal of Electronics Information Technology, 2013, 35(3): 627-632.

WELLING M. Fisher linear discriminant analysis[J]. Department of Computer Science, 2008, 16(94): 237-280.

FUIN N, PEDEMONTE S, ARRIDGE S, et al. Efficient determination of the uncertainty for the optimization of SPECT system design: a subsampled fisher information matrix[J]. IEEE Transactions on Medical Imaging, 2014, 33(3): 618-635.

DUFRENOIS F. A one-class kernel fisher criterion for outlier detection[J]. IEEE Transactions on Neural Networks Learning Systems, 2014, 26(5): 982-994.

VAN Ooyen A and NIENHUIS B. Improving the convergence of the back-propagation algorithm[J]. Neural Networks, 1992, 5(3): 465-471.

潘舟浩, 李道京, 刘波, 等. 基于BP算法和时变基线的机载InSAR数据处理方法研究[J]. 电子与信息学报, 2014, 36(7): 1585-1591.

PAN Zhouhao, LI Daojing, LIU Bo, et al. Processing of the airborne InSAR data based on the BP algorithm and the time-varying baseline[J] Journal of Electronics Information Technology, 2014, 36(7): 1585-1591.

SUYKENS J A K and VANDEWALLE J. Least squares support vector machine classifiers[J]. Neural Processing Letters, 1999, 9(3): 293-300.

胡文军, 王士同, 王娟, 等. 非线性分类的分割超平面快速集成方法[J]. 电子与信息学报, 2012, 34(3): 535-542.

HU Wenjun, WANG Shitong, WANG Juan, et al. Fast ensemble of separating hyperplanes for nonlinear classification[J]. Journal of Electronics Information Technology, 2012, 34(3): 535-542.

GAO X, LU T, LIU P, et al. A soil moisture classification model based on SVM used in agricultural WSN[C]. 2014 IEEE 7th Joint International, Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing, 2014: 432-436.

RIES C X, RICHTER F, ROMBERG S, et al. Automatic object annotation from weakly labeled data with latent structured SVM[C]. 2014 12th IEEE International Workshop on Content-based Multimedia Indexing (CBMI), Klagenfurt, Austria, 2014: 1-4.

PLATT J. Fast training of support vector machines using sequential minimal optimization[J]. Advances in Kernel Methods: Support Vector Learning, 1999(3): 185-208.

JOACHIMS T. Making large scale SVM learning practical[R]. Universit?t Dortmund, 1999.

CHANG C C and LIN C J. LIBSVM: A library for support vector machines[J]. ACM Transactions on Intelligent Systems Technology, 2011, 2(3): 389-396.

MANGASARIAN O L and MUSICANT D R. Lagrangian support vector machines[J]. The Journal of Machine Learning Research, 2001, 1(3): 161-177.

SEOK K. Semi-supervised regression based on support vector machine[J]. Computer Engineering Applications, 2014, 25(2): 447-454.

LENG Y, XU X, and QI G. Combining active learning and semi-supervised learning to construct SVM classifier[J]. Knowledge-Based Systems, 2013, 44(1): 121-131.

CHEN W J, SHAO Y H, XU D K, et al. Manifold proximal support vector machine for semi-supervised classification[J]. Applied Intelligence, 2014, 40(4): 623-638.

李红莲, 王春花, 袁保宗. 一种改进的支持向量机NN- SVM[J]. 计算机学报, 2003, 26(8): 1015-1020.

LI Honglian, WANG Chunhua, and YUAN Baozong. An improved SVM: NN-SVM[J]. Chinese Journal of Computers, 2003, 26(8): 1015-1020.

陈宝林. 最优化理论与算法[M]. 北京: 清华大学出版社, 2005: 281-322.

CHEN Baolin. Optimization Theory and Algorithm[M]. Beijing, Tsinghua University Press, 2005: 281-322.

YOSHIYAMA K and SAKURAI A. Laplacian minimax probability machine[J]. Pattern Recognition Letters, 2014, 37: 192-200.

MIGLIORATI G. Adaptive polynomial approximation by means of random discrete least squares[J]. Lecture Notes in Computational Science Engineering, 2013, 103: 547-554.

HUANG K, YANG H, KING I, et al. The minimum error minimax probability machine[J]. The Journal of Machine Learning Research, 2004(5): 1253-1286.

PLAN Y and VERSHYNIN R. Robust 1-bit compressed sensing and sparse logistic regression: A convex programming approach[J]. IEEE Transactions on Information Theory, 2013, 59(1): 482-494.

ALIZADEN F. Interior point methods in semidefinite programming with applications to combinatorial optimization[J]. SIAM Journal on Optimization, 1995, 5(1): 13-51.

BOYD S and VANDENBERGHE L. Convex Optimi- zation[M]. Cambridge University Press, 2009: 127- 214.

边肇祺, 张学工. 模式识别[M]. 北京: 清华大学出版社, 2001: 83-90.

BIAN Zhaoqi and ZHANG Xuegong. Pattern Recognition[M]. Beijing, Tsinghua University Press, 2001: 83-90.

Relative Articles

Supplements(0)

Cited By

Proportional views