Efficient multi-classifier wrapper feature-selection model: Application for dimension reduction in credit scoring

The task of identifying most relevant features for a credit scoring application is a challenging task. Reducing the number of redundant and unwanted features is an inevitable task to improve the performance of the credit scoring model. The wrappers approach is usually used in credit scoring applications to identify the most relevant features. However, this approach suffers from the issue of subsets generation and the use of a single classifier as an evaluation function. The problem here is that each classifier may give different results which can be interpreted differently. Hence, we propose in this study an ensemble wrapper feature selection model which is based on a multi-classifiers combination. In a first stage, we address the problem of subsets generation by minimizing the search space through a customized heuristic. Then, a multi-classifier wrapper evaluation is applied using two classifier arrangement approaches in order to select a set of mutually approved set of relevant features. The proposed method is evaluated on four credit datasets and has shown a good performance compared to individual classifiers results.