Premium
Pruning variable selection ensembles
Author(s) -
Zhang Chunxia,
Wu Yilei,
Zhu Mu
Publication year - 2019
Publication title -
statistical analysis and data mining: the asa data science journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.381
H-Index - 33
eISSN - 1932-1872
pISSN - 1932-1864
DOI - 10.1002/sam.11410
Subject(s) - computer science , benchmark (surveying) , selection (genetic algorithm) , sorting , ensemble learning , lasso (programming language) , feature selection , pruning , stability (learning theory) , artificial intelligence , context (archaeology) , machine learning , process (computing) , variable (mathematics) , pattern recognition (psychology) , algorithm , mathematics , paleontology , mathematical analysis , geodesy , world wide web , agronomy , biology , geography , operating system
In the context of variable selection, ensemble learning has gained increasing interest due to its great potential to improve selection accuracy and to reduce false discovery rate. A novel ordering‐based selective ensemble learning strategy is designed in this paper to obtain smaller but more accurate ensembles. In particular, a greedy sorting strategy is proposed to rearrange the order by which the members are included into the integration process. Through stopping the fusion process early, a smaller subensemble with higher selection accuracy can be obtained. More importantly, the sequential inclusion criterion reveals the fundamental strength‐diversity trade‐off among ensemble members. By taking stability selection with the base learner lasso (abbreviated as SSLasso) as an example, some experiments are conducted to examine the performance of the novel algorithm. Experimental results demonstrate that pruned SSLasso generally achieves higher selection accuracy and lower false discovery rate than SSLasso and several other benchmark methods.