Feature Selection Algorithm for Dynamically Weighted Conditional Mutual Information

Li ZHANG; Xiaobo CHEN

doi:10.11999/JEIT200615

Volume 43 Issue 10

Oct. 2021

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2021 > 43(10): 3028-3034

LI Yubo, CHEN Miao. Construction of Nearly Perfect Gaussian Integer Sequences[J]. Journal of Electronics & Information Technology, 2018, 40(7): 1752-1758. doi: 10.11999/JEIT170844

Citation:

Li ZHANG, Xiaobo CHEN. Feature Selection Algorithm for Dynamically Weighted Conditional Mutual Information[J]. Journal of Electronics & Information Technology, 2021, 43(10): 3028-3034. doi: 10.11999/JEIT200615

LI Yubo, CHEN Miao. Construction of Nearly Perfect Gaussian Integer Sequences[J]. Journal of Electronics & Information Technology, 2018, 40(7): 1752-1758. doi: 10.11999/JEIT170844

Citation:

PDF( 3607 KB)

Feature Selection Algorithm for Dynamically Weighted Conditional Mutual Information

doi: 10.11999/JEIT200615

Li ZHANG^{1, 2
,
,},
Xiaobo CHEN³

1.
College of Computer Engineering, Jiangsu University of Technology, Changzhou 213001, China
2.
Key Laboratory of Trustworthy Distributed Computing and Service (Ministry of Education), Beijing University of Posts and Telecommunications, Beijing 100876, China
3.
The People's Bank of China, Changzhou Branch, Changzhou 213001, China

Funds: The National Science and Technology Basic Work Project (2015FY111700-6), The Doctoral Research Fund of Jiangsu University of Technology (KYY19042)

Received Date: 2020-07-23
Rev Recd Date: 2021-02-05

Available Online: 2021-03-19

Publish Date: 2021-10-18

Abstract

Abstract

Feature selection is an essential step in the data preprocessing phase in the fields of machine learning, natural language processing and data mining. In some feature selection algorithms based on information theory, there is a problem that choosing different parameters means choosing different feature selection algorithms. How to determine the dynamic, non-a priori weights and avoid the preset a priori parameters become an urgent problem. A Dynamic Weighted Maximum Relevance and maximum Independence (WMRI) feature selection algorithm is proposed in this paper. Firstly, the algorithm calculates the average value of the new classification information and the retained classification information. Secondly, the standard deviation is used to dynamically adjust the parameter weights of these two types of classification information. At last, WMRI and the other five feature selection algorithms use ten different data sets on three classifiers for the fmi classification metrics validation. The experimental results show that the WMRI method can improve the quality of feature subsets and increase classification accuracy.
- Feature selection,
- Classification information,
- Average value,
- Standard deviation,
- Dynamic weighting

FullText(HTML)

References(22)

References

[1]	CHE Jinxing, YANG Youlong, LI Li, et al. Maximum relevance minimum common redundancy feature selection for nonlinear data[J]. Information Sciences, 2017, 409/410: 68–86. doi: 10.1016/j.ins.2017.05.013
[2]	CHEN Zhijun, WU Chaozhong, ZHANG Yishi, et al. Feature selection with redundancy-complementariness dispersion[J]. Knowledge-Based Systems, 2015, 89: 203–217. doi: 10.1016/j.knosys.2015.07.004
[3]	张天骐, 范聪聪, 葛宛营, 等. 基于ICA和特征提取的MIMO信号调制识别算法[J]. 电子与信息学报, 2020, 42(9): 2208–2215. doi: 10.11999/JEIT190320 ZHANG Tianqi, FAN Congcong, GE Wanying, et al. MIMO signal modulation recognition algorithm based on ICA and feature extraction[J]. Journal of Electronics &Information Technology, 2020, 42(9): 2208–2215. doi: 10.11999/JEIT190320
[4]	ZHANG Yishi, ZHANG Qi, CHEN Zhijun, et al. Feature assessment and ranking for classification with nonlinear sparse representation and approximate dependence analysis[J]. Decision Support Systems, 2019, 122: 113064.1–113064.17. doi: 10.1016/j.dss.2019.05.004
[5]	ZENG Zilin, ZHANG Hongjun, ZHANG Rui, et al. A novel feature selection method considering feature interaction[J]. Pattern Recognition, 2015, 48(8): 2656–2666. doi: 10.1016/j.patcog.2015.02.025
[6]	赵湛, 韩璐, 方震, 等. 基于可穿戴设备的日常压力状态评估研究[J]. 电子与信息学报, 2017, 39(11): 2669–2676. doi: 10.11999/JEIT170120 ZHAO Zhan, HAN Lu, FANG Zhen, et al. Research on daily stress detection based on wearable device[J]. Journal of Electronics &Information Technology, 2017, 39(11): 2669–2676. doi: 10.11999/JEIT170120
[7]	BROWN G, POCOCK A, ZHAO Mingjie, et al. Conditional likelihood maximisation: A unifying framework for information theoretic feature selection[J]. The Journal of Machine Learning Research, 2012, 13(1): 27–66.
[8]	MACEDO F, OLIVEIRA M R, PACHECO A, et al. Theoretical foundations of forward feature selection methods based on mutual information[J]. Neurocomputing, 2019, 325: 67–89. doi: 10.1016/j.neucom.2018.09.077
[9]	GAO Wanfu, HU Liang, ZHANG Ping, et al. Feature selection by integrating two groups of feature evaluation criteria[J]. Expert Systems with Applications, 2018, 110: 11–19. doi: 10.1016/j.eswa.2018.05.029
[10]	肖利军, 郭继昌, 顾翔元. 一种采用冗余性动态权重的特征选择算法[J]. 西安电子科技大学学报, 2019, 46(5): 155–161. doi: 10.19665/j.issn1001-2400.2019.05.022 XIAO Lijun, GUO Jichang, and GU Xiangyuan. Algorithm for selection of features based on dynamic weights using redundancy[J]. Journal of Xidian University, 2019, 46(5): 155–161. doi: 10.19665/j.issn1001-2400.2019.05.022
[11]	WANG Xinzheng, GUO Bing, SHEN Yan, et al. Input feature selection method based on feature set equivalence and mutual information gain maximization[J]. IEEE Access, 2019, 7: 151525–151538. doi: 10.1109/ACCESS.2019.2948095
[12]	GAO Wanfu, HU Liang, and ZHANG Ping. Class-specific mutual information variation for feature selection[J]. Pattern Recognition, 2018, 79: 328–339. doi: 10.1016/j.patcog.2018.02.020
[13]	WANG Jun, WEI Jinmao, YANG Zhenglu, et al. Feature selection by maximizing independent classification information[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(4): 828–841. doi: 10.1109/TKDE.2017.2650906
[14]	GAO Wanfu, HU Liang, ZHANG Ping, et al. Feature selection considering the composition of feature relevancy[J]. Pattern Recognition Letters, 2018, 112: 70–74. doi: 10.1016/j.patrec.2018.06.005
[15]	LIN Xiaohui, LI Chao, REN Weijie, et al. A new feature selection method based on symmetrical uncertainty and interaction gain[J]. Computational Biology and Chemistry, 2019, 83: 107149. doi: 10.1016/j.compbiolchem.2019.107149
[16]	BENNASAR M, HICKS Y, and SETCHI R. Feature selection using joint mutual information maximisation[J]. Expert Systems with Applications, 2015, 42(22): 8520–8532. doi: 10.1016/j.eswa.2015.07.007
[17]	LYU Hongqiang, WAN Mingxi, HAN Jiuqiang, et al. A filter feature selection method based on the maximal information coefficient and Gram-Schmidt orthogonalization for biomedical data mining[J]. Computers in Biology and Medicine, 2017, 89: 264–274. doi: 10.1016/j.compbiomed.2017.08.021
[18]	SHARMIN S, SHOYAIB M, ALI A A, et al. Simultaneous feature selection and discretization based on mutual information[J]. Pattern Recognition, 2019, 91: 162–174. doi: 10.1016/j.patcog.2019.02.016
[19]	SUN Guanglu, LI Jiabin, DAI Jian, et al. Feature selection for IoT based on maximal information coefficient[J]. Future Generation Computer Systems, 2018, 89: 606–616. doi: 10.1016/j.future.2018.05.060
[20]	PENG Hanchuan, LONG Fuhui, and DING C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(8): 1226–1238. doi: 10.1109/TPAMI.2005.159
[21]	MEYER P E, SCHRETTER C, and BONTEMPI G. Information-theoretic feature selection in microarray data using variable complementarity[J]. IEEE Journal of Selected Topics in Signal Processing, 2008, 2(3): 261–274. doi: 10.1109/JSTSP.2008.923858
[22]	SPEISER J L, MILLER M E, TOOZE J, et al. A comparison of random forest variable selection methods for classification prediction modeling[J]. Expert Systems with Applications, 2019, 134: 93–101. doi: 10.1016/j.eswa.2019.05.028

Relative Articles

Supplements(0)

Cited By

Cited by

Periodical cited type(19)

1.	孙顺远，魏志涛. 基于二次移动平均法估计背景光照的二值化方法. 计算机与数字工程. 2024(06): 1830-1836 .
2.	赵孔卫，徐广标. 基于像素分析的针织面料卷边性评价研究. 针织工业. 2024(10): 11-14 .
3.	卢晓波，徐海，朱俊召，张宇，谭健，高冠男，胡军华，林龙. 基于机器视觉的加热卷烟烟支端部质量检测系统设计. 轻工学报. 2024(06): 101-107+115 .
4.	韩海豹，化荣，张虎，陈杰. 量产活禽（肉鸡）智能化运输装备控制系统的设计. 农业技术与装备. 2023(01): 20-22 .
5.	支亚京，汤宁，吴兴洋，汪华，胡兴炜，张军. 基于支持向量机的气温自记纸图像数字化. 计算机技术与发展. 2023(10): 216-220 .
6.	魏兴凯，蒋峥，傅呈勋，刘斌. 基于光照影响因子的动态Niblack算法研究及应用. 计算机工程与设计. 2022(04): 1066-1073 .
7.	徐浩，章明希. 高精密齿轮小缺陷的智能视觉测量. 兵器材料科学与工程. 2021(01): 83-87 .
8.	贺欢，吐尔洪江·阿布都克力木，何笑. 一种基于MALLAT算法的图像去雾方法. 新疆师范大学学报(自然科学版). 2020(01): 23-27 .
9.	赵琛，张血琴，刘凯，郭裕钧. 基于正则化的多光谱图像二值化处理. 计算机仿真. 2020(04): 436-440 .
10.	杜炤鑫，谢海宁，宋杰，周德生，邹晓峰，陈冉，曾平. 基于图像处理和深度学习的配网跳闸故障识别方法. 中国科学技术大学学报. 2020(01): 39-48 .
11.	蒋鹏程，熊礼治，韩啸. 一种基于内容保护与优化识别的二维码方案. 软件导刊. 2019(02): 119-122 .
12.	安建尧，李金新，孙双平. 基于Prewitt算子的红外图像边缘检测改进算法. 杭州电子科技大学学报(自然科学版). 2018(05): 18-23+39 .
13.	陈志伟，徐世许，刘云鹏，曾祥晓. 基于视觉筛选的并联机器人平面抓取系统设计. 制造业自动化. 2018(05): 44-47 .
14.	熊炜，徐晶晶，赵诗云，王改华，刘敏，赵楠，刘聪. 基于支持向量机的低质量文档图像二值化. 计算机应用与软件. 2018(02): 218-223+241 .
15.	李昌利，周晓晓，张振，樊棠怀. Retinex模型下基于融合策略的雾霾图像增强. 工程科学与技术. 2018(05): 202-208 .
16.	于晓，闫振雷，周子杰. 指纹识别网页登录器设计. 实验室研究与探索. 2018(10): 85-88+128 .
17.	宋巧君，张东. 基于双边滤波和Black-hat变换的OSTU裂缝分割算法. 信息技术. 2017(12): 90-92 .
18.	谢芳娟，曾萍萍，谭菊华. 低分辨率灰度图像传输真实度优化仿真研究. 计算机仿真. 2017(12): 183-186 .
19.	田敬波. 基于模板算子边缘检测的图像二值化算法. 信息技术与信息化. 2017(09): 98-101 .