融合改进二元萤火虫算法和互补性测度的集成剪枝方法

朱旭辉; 倪志伟; 倪丽萍; 金飞飞; 程美英; 李敬明

doi:10.11999/JEIT170984

融合改进二元萤火虫算法和互补性测度的集成剪枝方法

doi: 10.11999/JEIT170984

^①(合肥工业大学管理学院合肥 230009) ^②(过程优化与智能决策教育部重点实验室合肥 230009) ^③(湖州师范学院商学院湖州 313000) ^④(安徽财经大学管理科学与工程学院蚌埠 233030)

基金项目:

国家自然科学基金(91546108, 71271071, 71490725, 71301041)，国家重点研发计划(2016YFF0202604)，过程优化与智能决策教育部重点实验室开放课题

详细信息

作者简介:
朱旭辉：男，1991年生，博士生，研究方向为进化计算和机器学习. 倪志伟：男，1963年生，教授，研究方向为人工智能、机器学习和云计算. 倪丽萍：女，1981年生，副教授，研究方向为分形数据挖掘、人工智能和机器学习. 金飞飞：男，1988年生，博士生，研究方向为智能决策和智能计算. 程美英：女，1983年生，讲师，研究方向为智能计算和数据挖掘. 李敬明：男，1979年生，讲师，研究方向为智能计算和数据挖掘.

中图分类号: TP391
计量
- 文章访问数: 1208
- HTML全文浏览量: 224
- PDF下载量: 55
- 被引次数: 0
出版历程
- 收稿日期: 2017-10-23
- 修回日期: 2018-04-02
- 刊出日期: 2018-07-19

Improved Binary Glowworm Swarm Optimization Combined with Complementarity Measure for Ensemble Pruning

ZHU Xuhui^①② NI Zhiwei^①② NI Liping^①② JIN Feifei^①② CHENG Meiying^③ LI Jingming^④

Funds:

The National Natural Science Foundation of China (91546108, 71271071, 71490725, 71301041), The National Key Research and Development Plan (2016YFF0202604), Open Research Fund Program of Key Laboratory of Process Optimization and Intelligent Decision-making

摘要

摘要: 差异性和平均精度是提高分类器集成性能的两个重要指标。增加差异性势必会降低平均精度，增大平均精度一定会减小差异性。故在差异性和平均精度之间存在一个平衡状态，使得集成性能最优。为了寻找该平衡状态，该文提出融合改进二元萤火虫算法和互补性测度的集成剪枝方法。首先，采用bootstrap抽样方法独立训练出多个基分类器，构建原始基分类器池。其次，采用互补性测度对原始基分类器池进行预剪枝。接着，通过改进萤火虫的移动方式和搜索过程，引入重新初始化机制和跳跃行为，提出改进二元萤火虫算法。最后，采用改进二元萤火虫算法对预剪枝后的基分类器，进行进一步剪枝，选择出集成性能最优的基分类器子集合。在5个UCI数据集上的实验结果表明，较其他方法，使用较少的基分类器，获得了更优的集成性能，具有良好的有效性和显著性。
- 萤火虫算法 /
- 互补性测度 /
- 集成剪枝
Abstract: The key to the success of an ensemble system are the diversity and the average accuracy of base classifiers. The increase of diversity among base classifiers will lead to the decrease of the average accuracy, and vice versa. So there exists a tradeoff between the diversity and the average accuracy, which makes the ensemble perform the best with respect to ensemble pruning. To find the tradeoff, Improved Binary Glowworm Swarm Optimization combined with Complementarity measure for Ensemble Pruning (IBGSOCEP) is proposed. Firstly, an initial pool of classifiers is constructed through training independently some base classifiers using bootstrap sampling. Secondly, the classifiers in the initial pool are pre-pruned using complementarity measure. Thirdly, Improved Binary Glowworm Swarm Optimization (IBGSO) is proposed by improving moving way, searching processes of glowworm, introducing re-initialization, and leaping behaviors. Finally, the optimal sub-ensemble is achieved from the base classifiers after pre-pruning using IBGSO. Experimental results on 5 UCI datasets demonstrate that IBGSODSEN can achieve better results than other approaches with less number of base classifiers, and that its effectiveness and significance.
- Glowworm Swarm Optimization (GSO)、Complementarity measure、Ensemble pruning /

HTML全文

参考文献(20)

BASHBAGHI S, GRANGER E, SABOURIN R, et al. Dynamic ensembles of exemplar-SVMs for still-to-video face recognition[J]. Pattern Recognition, 2017, 69(C): 61-81. doi: 10.1016/j.patcog.2017.04.014.

LIU Jiachen, MIAO Qiguang, CAO Ying, et al. Ensemble one-class classifiers based on hybrid diversity generation and pruning[J]. Journal of Electronics & Information Technology, 2015, 37(2): 386-393. doi: 10.11999/JEIT140161.

[3] LI Kai, XING Junliang, HU Weiming, et al. D2C: Deep cumulatively and comparatively learning for human age estimation[J]. Pattern Recognition, 2017, 66(6): 95-105. doi: 10.1016/j.patcog.2017.01.007.

[4] LU Huijuan, AN Chunlin, ZHENG Enhui, et al. Dissimilarity based ensemble of extreme learning machine for gene expression data classification[J]. Neurocomputing, 2014, 128(5): 22-30. doi: 10.1016/j.neucom.2013.02.052.

YANG Chun, YIN Xucheng, HAO Hongwei, et al. Classifier ensemble with diversity: Effectiveness analysis and ensemble optimization[J]. Acta Automatica Sinica, 2014, 40(4): 660-674. doi: 10.3724/SP.J.1004.2014.00660.

[6] YKHLEF H and BOUCHAFFRA D. An efficient ensemble pruning approach based on simple coalitional games[J]. Information Fusion, 2017, 34(C): 28-42. doi: 10.1016/j.inffus. 2016.06.003.

[7] ZHOU Zhihua, WU Jianxin, and TANG Wei. Ensembling neural networks: many could be better than all[J]. Artificial Intelligence, 2002, 137(1): 239-263. doi: 10.1016/S0004- 3702(02)00190-X.

[8] MARTÍNEZ-MUÑOZ G, HERNANDEZ LOBATO D, and SUAREZ A. An analysis of ensemble pruning techniques based on ordered aggregation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(2): 245-259. doi: 10.1109/TPAMI.2008.78.

[9] GUO Li and BOUKIR S. Margin-based ordered aggregation for ensemble pruning[J]. Pattern Recognition Letters, 2013, 34(6): 603-609. doi: 10.1016/j.patrec.2013.01.003.

[10] DAI Qun, ZHANG Ting, and LIU Ningzhong. A new reverse reduce-error ensemble pruning algorithm[J]. Applied Soft Computing, 2015, 28(3): 237-249. doi: 10.1016/j.asoc.2014. 10.045.

[11] ROKACH L. Collective-agreement-based pruning of ensembles[J]. Computational Statistics and Data Analysis, 2009, 53(4): 1015-1026. doi: 10.1016/j.csda.2008.12.001.

[12] LAZAREVIC A and OBRADOVIC Z. Effective pruning of neural network classifier ensembles[C]. International Joint Conference on Neural Networks, Washington DC, USA, 2001: 796-801. doi: 10.1109/IJCNN.2001.939461.

[13] GIACINTO G, ROLI F, and FUMERA G. Design of effective multiple classifier systems by clustering of classifiers[C]. International Conference on Pattern Recognition, Barcelona, Spain, 2000: 160-163. doi: 10.1109/ICPR.2000.906039.

[14] BAKKER B and HESKES T. Clustering ensembles of neural network models[J]. Neural Network, 2003, 16(2): 261-269. doi: 10.1016/S0893-6080(02)00187-9.

[15] ZHOU Hongfang, ZHAO Xuehan, and WANG Xiao. An effective ensemble pruning algorithm based on frequent patterns[J]. Knowledge-Based Systems, 2014, 56(C): 79-85. doi: 10.1016/j.knosys.2013. 10.024.

[16] CAVALCANTI G D C, OLIVEIRA L S, NOURA T J M, et al. Combining diversity measures for ensemble pruning[J]. Pattern Recognition Letters, 2016, 74(C): 38-45. doi: 10.1016 /j.patrec.2016.01.029.

NI Zhiwei, ZHANG Chen, and NI Liping. Haze forecast method of selective ensemble based on glowworm swarm optimization algorithm[J]. Pattern Recognition and Artificial Intelligence, 2016, 29(2): 143-153. doi: 10.16451/j.cnki. issn1003-6059.201602006.

[18] MARINAKI M and MARINAKI Y. A glowworm swarm optimization algorithm for the vehicle routing problem with stochastic demands[J]. Expert Systems with Applications, 2016, 46(C): 145-163. doi: 10.1016/j.eswa.2015.10.012.

[19] BREIMAN L. Bagging predictors[J]. Machine Learning, 1996, 24(2): 123-140. doi: 10.1023/A:1018054314350.

[20] SINGHAL P K, NARESH R, and SHARMA V. Binary fish swarm algorithm for profit-based unit commitment problem in competitive electricity market with ramp rate constraints [J]. IET Generation, Transmission & Distribution, 2015, 9(13): 1697-1707. doi: 10.1049/iet-gtd.2015.0201.

施引文献

资源附件(0)

访问统计