Open AccessConditional Mutual Information‐Based Feature Selection Analyzing for Synergy and Redundancy
Author(s) -
Cheng grong,
Qin Zhiguang,
Feng Chaosheng,
Wang Yong,
Li Fagen
Publication year - 2011
Publication title -
etri journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.295
H-Index - 46
eISSN - 2233-7326
pISSN - 1225-6463
DOI - 10.4218/etrij.11.0110.0237
Subject(s) - mutual information , redundancy (engineering) , feature selection , discriminative model , minimum redundancy feature selection , computer science , conditional mutual information , artificial intelligence , pattern recognition (psychology) , feature (linguistics) , data mining , interaction information , machine learning , mathematics , philosophy , statistics , operating system , linguistics
Battiti's mutual information feature selector (MIFS) and its variant algorithms are used for many classification applications. Since they ignore feature synergy, MIFS and its variants may cause a big bias when features are combined to cooperate together. Besides, MIFS and its variants estimate feature redundancy regardless of the corresponding classification task. In this paper, we propose an automated greedy feature selection algorithm called conditional mutual information‐based feature selection (CMIFS). Based on the link between interaction information and conditional mutual information, CMIFS takes account of both redundancy and synergy interactions of features and identifies discriminative features. In addition, CMIFS combines feature redundancy evaluation with classification tasks. It can decrease the probability of mistaking important features as redundant features in searching process. The experimental results show that CMIFS can achieve higher best‐classification‐accuracy than MIFS and its variants, with the same or less (nearly 50%) number of features.