Using evolution to improve neural network learning: pitfalls and solutions

Autonomous neural network systems typically require fast learning and good generalization performance, and there is potentially a trade-off between the two. The use of evolutionary techniques to improve the learning abilities of neural network systems is now widespread. However, there are a range of different evolutionary approaches that could be applied, and no systematic investigation has been carried out to find which work best. In this paper, such an investigation is presented, and it is shown that a range of evolutionary techniques can generate high performance networks, but they often lead to unwanted side effects, such as occasional instances of very poor performance. The nature of these problems are explored further, and it is shown how the evolution of age dependent plasticities and/or the use of ensemble techniques can alleviate them. A range of techniques are thus identified, with differing properties, that can be matched to the specific requirements of each application.