v-Soft Margin Logistic Regression Classifier

HUANG Chengquan; WANG Shitong; JIANG Yizhang; DONG Aimei

doi:10.11999/JEIT150769

Volume 38 Issue 4

Apr. 2016

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2016 > 38(4): 985-992

HUANG Chengquan, WANG Shitong, JIANG Yizhang, DONG Aimei. v-Soft Margin Logistic Regression Classifier[J]. Journal of Electronics & Information Technology, 2016, 38(4): 985-992. doi: 10.11999/JEIT150769

Citation:

HUANG Chengquan, WANG Shitong, JIANG Yizhang, DONG Aimei. v-Soft Margin Logistic Regression Classifier[J]. Journal of Electronics & Information Technology, 2016, 38(4): 985-992. doi: 10.11999/JEIT150769

Citation:

PDF( 1019 KB)

v-Soft Margin Logistic Regression Classifier

doi: 10.11999/JEIT150769

1.
2.
(School of Digital Media, Jiangnan University, Wuxi 214122, China)

Funds:

The National Natural Science Foundation of China (61272210, 61202311), The Natural Science Foundation of Jiangsu Province (BK2012552), The Science and Technology Foundation of Guizhou Province ([2013]2136)

Received Date: 2015-06-29
Rev Recd Date: 2015-12-08
Publish Date: 2016-04-19

Abstract

Abstract

Coordinate Descent (CD) is a promising method for large scale pattern classification issues with straightforward operation and fast convergence speed. In this paper, inspired by v-soft margin Support Vector Machine (v-SVM) for pattern classification, a new v-Soft Margin Logistic Regression Classifier (v-SMLRC) is proposed for pattern classification to enhance the generalization performance of Logistic Regression Classifier (LRC). The dual of v-SMLRC can be regarded as CDdual problem with equality constraint and then a new large scale pattern classification method called v-SMLRC-CDdual is proposed. The proposed v-SMLRC-CDdual can maximize the inner-class margin and effectively enhance the generalization performance of LRC. Empirical results conducted with large scale document datasets demonstrate that the proposed method is effective and comparable to other related methods.
- Logistic regression,
- Generalization,
- Coordinate Descent (CD),
- Dual Coordinate Descent (CDdual)

FullText(HTML)

References(27)

References

BOTTOU L and BOUSQUET O. The tradeoffs of large scale learning[C]. Proceedings of Advances in Neural Information Processing Systems, Cambridge, 2008: 151-154.

LIN C Y, TSAI C H, LEE C P, et al. Large-scale logistic regression and linear support vector machines using Spark[C]. Proceedings of 2014 IEEE International Conference on Big Data, Washington DC, 2014: 519-528. doi: 10.1109/BigData. 2014.7004269.

AGERRI R, ARTOLA X, BELOKI Z, et al. Big data for natural language processing: A streaming approach[J]. Knowledge-Based Systems, 2015, 79: 36-42. doi: 10.1016/ j.knosys.2014.11.007.

DARROCH J N and RATCLIFF D. Generalized iterative scaling for log-linear models[J]. The Annals of Mathematical Statistics, 1972, 43(5): 1470-1480. doi: 10.1214/aoms/ 1177692379.

DELLA P S, DELLA P V, and LAFFERTY J. Inducing features of random fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(4): 380-393. doi: 10.1109/34.588021.

GOODMAN J. Sequential conditional generalized iterative scaling[C]. Proceedings of the 40th annual meeting of the association of computational linguistics, Philadelphia, 2002: 9-16. doi: 10.3115/1073083.1073086.

JIN R, YAN R, ZHANG J, et al. A faster iterative scaling algorithm for conditional exponential model[C]. Proceedings of the 20th International Conference on Machine Learning, New York, 2003: 282-289.

HUANG F L, HSIEN C J, CHANG K W, et al. Iterative scaling and coordinate descent methods for maximum entropy[J]. Journal of Machine Learning Research, 2010, 11(2): 815-848.

KOMAREK P and MOORE A W. Making logistic regression a core data mining tool: a practical investigation of accuracy, speed, and simplicity[R]. Technical report TR-05-27, Robotics Institute of Carnegie Mellon University, Pittsburgh, 2005.

LIN C J, WENG R C, and KEERTHI S S. Trust region Newton method for large-scale logistic regression[J]. Journal of Machine Learning Research, 2008, 9(4): 627-650.

KEERTHI S S, DUAN K B, SHEVADE S K, et al. A fast dual algorithm for kernel logistic regression[J]. Machine Learning, 2005, 61(1-3): 151-165. doi: 10.1007/s10994- 005-0768-5.

PLATT J C. Fast training of support vector machines using sequential minimal optimization[C]. Proceedings of Advances in Kernel Methods: Support Vector Learning, Cambridge, 1999: 185-208.

YU H F, HUANG F L, and LIN C J. Dual coordinate descent methods for logistic regression and maximum entropy models[J]. Machine Learning, 2011, 85(1/2): 41-75. doi: 10.1007/s10994-010-5221-8.

顾鑫, 王士同, 许敏. 基于多源的跨领域数据分类快速新算法[J]. 自动化学报, 2014, 40(3): 531-547. doi: 10.3724/SP.J. 1004.2014.00531.

GU X, WANG S T, and XU M. A new cross-multidomain classification algorithm and its fast version for large datasets[J]. Acta Automatica Sinica, 2014, 40(3): 531-547. doi: 10.3724/SP.J.1004.2014.00531.

顾鑫, 王士同. 大样本多源域与小目标域的跨领域快速分类学习[J]. 计算机研究与发展, 2014, 51(3): 519-535. doi: 10.7544/issn1000-1239.2014.20120652.

GU X and WANG S T. Fast cross-domain classification method for large multisources/small target domains[J]. Journal of Computer Research and Development, 2014, 51(3): 519-535. doi: 10.7544/issn1000-1239.2014.20120652.

张学峰, 陈渤, 王鹏辉, 等. 一种基于Dirichelt过程隐变量支撑向量机模型的目标识别方法[J]. 电子与信息学报, 2015, 37(1): 29-36. doi: 10.11999/JEIT140129.

ZHANG X F, CHEN B, WANG P H, et al. A target recognition method based on dirichlet process latent variable support vector machine model[J]. Journal of Electronics Information Technology, 2015, 37(1): 29-36. doi: 10.11999/ JEIT140129.

及歆荣, 侯翠琴, 侯义斌. 无线传感器网络下线性支持向量机分布式协同训练方法研究[J]. 电子与信息学报, 2015, 37(3): 708-714. doi: 10.11999/JEIT140408.

JI X R, HOU C Q, and HOU Y B. Research on the distributed training method for linear SVM in WSN[J]. Journal of Electronics Information Technology, 2015, 37(3): 708-714. doi: 10.11999/JEIT140408.

高发荣, 王佳佳, 席旭刚, 等. 基于粒子群优化-支持向量机方法的下肢肌电信号步态识别[J]. 电子与信息学报, 2015, 37(5): 1154-1159. doi: 10.11999/JEIT141083.

GAO F R, WANG J J, XI X G, et al. Gait recognition for lower extremity electromyographic signals based on PSO- SVM method[J]. Journal of Electronics Information Technology, 2015, 37(5): 1154-1159. doi: 10.11999/ JEIT141083.

HSIEH C J, CHANG K W, LIN C J, et al. A dual coordinate descent method for large-scale linear SVM[C]. Proceedings of the 25th International Conference on Machine Learning, New York, 2008: 408-415. doi: 10.1145/1390156.1390208.

CHEN P H, LIN C J, and SCHLKOPF B. A tutorial on v-support vector machines[J]. Applied Stochastic Models in Business and Industry, 2005, 21(2): 111-136. doi: 10.1002/ asmb.537.

PENG X J, CHEN D J, and KONG L Y. A clipping dual coordinate descent algorithm for solving support vector machines[J]. Knowledge-Based Systems, 2014, 71: 266-278. doi: 10.1016/j.knosys.2014.08.005.

TSAI C H, LIN C Y, and LIN C J. Incremental and decremental training for linear classification[C]. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, 2014: 343-352. doi: 10.1145/2623330.2623661.

Relative Articles

Supplements(0)

Cited By

Proportional views