A Learning Algorithm For Neural Network Ensembles

The performance of a single regressor/classifier can be improved by combining the outputs of several predictors. This is true provided the combined predictors are accurate and diverse enough, which posses the problem of generating suitable aggregate members in order to have optimal generalization capabilities. We propose here a new method for selecting members of regression/classification ensembles. In particular, using artificial neural networks as learners in a regression context, we show that this method leads to small aggregates with few but very diverse individual networks. The algorithm is favorably tested against other methods recently proposed in the literature, producing equal performance on the standard statistical databases used as benchmarks with ensembles that have 75% less members on average. Keywords: Neural Networks, Ensemble Learning, Regression, Bias/Variance Decomposition.1. Introduction Ensemble techniques have been used recently in regression/classification ,tasks ,with ,considerable success. The motivation for this procedure is based