基于有序编码的核极限学习顺序回归模型

李佩佳; 石勇; 汪华东; 牛凌峰

doi:10.11999/JEIT170765

基于有序编码的核极限学习顺序回归模型

doi: 10.11999/JEIT170765

基金项目:

国家自然科学基金(71110107026, 71331005, 91546201, 11671379, 111331012)，中国科学院大学资助项目(Y55202LY00)

计量
- 文章访问数: 1559
- HTML全文浏览量: 161
- PDF下载量: 173
- 被引次数: 0
出版历程
- 收稿日期: 2017-07-28
- 修回日期: 2018-01-22
- 刊出日期: 2018-06-19

Ordered Code-based Kernel Extreme Learning Machine for Ordinal Regression

Funds:

The National Natural Science Foundation of China (71110107026, 71331005, 91546201, 11671379, 111331012), The Grant of University of Chinese Academy of Sciences (Y55202LY00)

摘要

摘要: 顺序回归是机器学习领域中介于分类和回归之间的有监督问题。在实际中，许多带有序关系标签的问题都可以被建模成顺序回归问题，因此顺序回归受到众多学者的关注。基于极限学习机(ELM)的算法能有效避免因迭代过程陷入的局部最优解，减少训练时间，但基于极限学习机的算法在顺序回归问题上的研究较少。该文将核极限学习机与纠错输出编码相结合，提出了一种基于有序编码的核极限学习顺序回归模型。该模型有效解决了如何在顺序回归中取得良好的特征映射以及如何避免传统极限学习机中隐层节点个数依赖于人工设置的问题。为验证提出模型的有效性，该文在多个顺序回归数据集上进行了测试，测试结果表明，相比于传统ELM模型，该文提出的模型在准确率上平均提升了10.8%，在数据集上预测表现最优，而且获得了最短的训练时间，从而验证了模型的有效性。
- 纠错输出编码 /
- 顺序回归 /
- 极限学习机 /
- 核函数
Abstract: Ordinal regression is one of the supervised learning issues, which resides between classification and regression in machine learning fields. There exist many real problems in practice, which can be modeled as ordinal regression problems due to the ordering information between labels. Therefore ordinal regression has received increasing interest by many researchers recently. The Extreme Learning Machine (ELM)-based algorithms are easy to train without iterative algorithm and they can avoid the local optimal solution; meanwhile they reduce the training time compared with other learning algorithms. However, the ELM-based algorithms which are applied to ordinal regression have not been exploited much. This paper proposes a new ordered code-based kernel extreme learning ordinal regression machine to fill this gap, which combines the kernel ELM and error correcting output codes effectively. The model overcomes the problems of how to get high quality feature mappings in ordinal regression and how to avoid setting the number of hidden nodes by manual. To validate the effectiveness of this model, numerous experiments are conducted on a lot of datasets. The experimental results show that the model can improve the accuracy by 10.8% on average compared with traditional ELM-based algorithms and achieve the state- of-the-art performance with the least time.
- Error correct output code /
- Ordinal regression /
- Extreme learning machine /
- Kernel function

HTML全文

参考文献(30)

NAKOV P, RITTER A, ROSENTHAL S, et al. SemEval- 2016 task 4: Sentiment analysis in Twitter[C]. International Workshop on Semantic Evaluation, San Diego, USA, 2016: 1-18. doi: 10.18653/v1/S16-1028.

TIAN Q, CHEN S, and TAN X. Comparative study among three strategies of incorporating spatial structures to ordinal image regression[J]. Neurocomputing, 2014, 136: 152-161. doi: 10.1016/j.neucom.2014.01.017.

CORRENTE S, DOUMPOS M, GRECO S, et al. Multiple criteria hierarchy process for sorting problems based on ordinal regression with additive value functions[J]. Annals of Operations Research, 2017, 251(1/2): 117-139. doi: 10.1007/ s10479-015-1898-1.

GUTIRREZ P A, PREZ-ORTIZ M, SANCHEZ- MONEDERO J, et al. Ordinal regression methods: Survey and experimental study[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(1): 127-146. doi: 10.1109/ TKDE.2015.2457911.

HUANG G B, ZHU Q Y, and SIEW C K. Extreme learning machine: Theory and applications[J]. Neurocomputing, 2006, 70(1): 489-501. doi: 10.1016/j.neucom.2005.12.126.

RAJASEKARAN S and PAI G A V. Neural Networks, Fuzzy Systems and Evolutionary Algorithms: Synthesis and Applications[M]. Haryana, India: Rajkamal Electric Press, 2017: 151-168.

CHU W and KEERTHI S S. Support vector ordinal regression[J]. Neural Computation, 2007, 19(3): 792-815. doi: 10.1162/neco.2007.19.3.792.

HUANG G B, ZHOU H, DING X, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42(2): 513-529. doi: 10.1109/TSMCB. 2011.2168604.

UCAR A, DEMIR Y, and GZELI C. A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering[J]. Neural Computing and Applications, 2016, 27(1): 131-142. doi: 10.1007/s00521-014-1569-1.

徐涛, 郭威, 吕宗磊. 基于快速极限学习机和差分进化的机场噪声预测模型[J]. 电子与信息学报, 2016, 38(6): 1512-1518. doi: 10.11999/JEIT150986.

XU Tao, GUO Wei, and L Zonglei. Prediction model of airport noise based on fast extreme learning machine and differential evolution[J]. Journal of Electronics Information Technology, 2016, 38(6): 1512-1518. doi: 10.11999/JEIT 150986.

GOODFELLOW I, BENGIO Y, and COURVILLE A. Deep Learning[M]. Massachusetts, USA, MIT Press, 2016: 165-480. doi: 10.1038/nature14539.

DENG W Y, ZHENG Q H, LIAN S, et al. Ordinal extreme learning machine[J]. Neurocomputing, 2010, 74(1): 447-456. doi: 10.1016/j.neucom.2010.08.022.

RICCARDI A, FERNNDEZ-NAVARRO F, and CARLONI S. Cost-sensitive AdaBoost algorithm for ordinal regression based on extreme learning machine[J]. IEEE Transactions on Cybernetics, 2014, 44(10): 1898-1909. doi: 10.1109/TCYB. 2014.2299291.

HORNIK K, STINCHCOMBE M, and WHITE H. Multilayer feedforward networks are universal approximators[J]. Neural Networks, 1989, 2(5): 359-366. doi: 10.1016/0893-6080(89) 90020-8.

HUANG G B and BABRI H A. Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions[J]. IEEE Transactions on Neural Networks, 1998, 9(1): 224-229. doi: 10.1109/72.655045.

HUANG G B, CHEN L, and SIEW C K. Universal approximation using incremental constructive feedforward networks with random hidden nodes[J]. IEEE Transactions on Neural Networks, 2006, 17(4): 879-892. doi: 10.1109/TNN. 2006.875977.

HUANG G B. Learning capability and storage capacity of two-hidden-layer feedforward networks[J]. IEEE Transactions on Neural Networks, 2003, 14(2): 274-281. doi: 10.1109/TNN.2003.809401.

BARTLETT P L. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network[J]. IEEE Transactions on Information Theory, 1998, 44(2): 525-536. doi: 10.1109/18.661502.

TANG J, DENG C, and HUANG G B. Extreme learning machine for multilayer perceptron[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(4): 809-821. doi: 10.1109/TNNLS.2015.2424995.

HOERL A E and KENNARD R W. Ridge regression: Biased estimation for nonorthogonal problems[J]. Technometrics, 1970, 12(1): 55-67. doi: 10.1080/00401706.1970.10488634.

ALLWEIN E L, SCHAPIRE R E, and SINGER Y. Reducing multiclass to binary: A unifying approach for margin classifiers[J]. Journal of Machine Learning Research, 2000, 1(12): 113-141. doi: 10.1162/15324430152733133.

雷蕾, 王晓丹, 罗玺, 等. ECOC多类分类研究综述[J]. 电子学报, 2014, 42(9): 1794-1800. doi: 10.3969/j.issn.0372-2112. 2014.09.020.

LEI Lei, WANG Xiaodan, LUO Xi, et al. An overview of multi-classification based on error-correcting output codes[J]. Acta Electronica Sinica, 2014, 42(9): 1794-1800. doi: 10.3969 /j.issn.0372-2112.2014.09.020.

HUANG G, HUANG G B, SONG S, et al. Trends in extreme learning machines: A review[J]. Neural Networks, 2015, 61: 32-48. doi: 10.1016/j.neunet.2014.10.001.

LIU Q, HE Q, and SHI Z. Extreme support vector machine classifier[C]. 12th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Osaka, Japan, 2008: 222-233. doi: 10.1007/978-3-540-68125-0_21.

FRNAY B and VERLEYSEN M. Using SVMs with randomised feature spaces: an extreme learning approach[C]. European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium, 2010: 315-320.

HUANG G B, DING X, and ZHOU H. Optimization method based extreme learning machine for classification[J]. Neurocomputing, 2010, 74(1): 155-163. doi: 10.1016/j.neucom. 2010.02.019.

CHU W and GHAHRAMANI Z. Gaussian processes for ordinal regression[J]. Journal of Machine Learning Research, 2005, 6(7): 1019-1041.

BACCIANELLA S, ESULI A, and SEBASTIANI F. Evaluation measures for ordinal regression[C]. The Ninth International Conference on Intelligent Systems Design and Applications, Pisa, Italy, 2009: 283-287. doi: 10.1109/ISDA. 2009.230.

施引文献

资源附件(0)

访问统计