A Minimal Neural Network Ensemble Construction Method: A Constructive Approach

This paper presents a neural network ensemble (NNE) construction method for classification problems. The proposed method automatically determines a minimal NNE architecture and thus called the Minimal Neural Network Ensemble Construction (MNNEC) method. To determine minimal architecture, it starts with a single neural network (NN) with a minimal number of hidden units. During training process, it adds additional NN(s) with cumulative number(s) of hidden units. In conventional methods, in contrast, the number of NNs for NNE and the number of hidden nodes for each NN should be predetermined. At the time of NN addition in MNNEC, the added NN specializes in the previously unsolved portion of the input space. Finally all the NNs are trained simultaneously to improve the generalization ability. Therefore, for easy problems when multiple NNs are not required and a single NN is sufficient, the MNNEC can generate a single NN with a minimal number of hidden units. The MNNEC has been tested extensively on several benchmark problems of machine learning and NNs. The results exhibit that the MNNEC is able to construct NNEs of much smaller size than conventional methods.