Premium
Improved Particle Swarm Optimization Combined with Backpropagation for Feedforward Neural Networks
Author(s) -
Han Fei,
Zhu JianSheng
Publication year - 2013
Publication title -
international journal of intelligent systems
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.291
H-Index - 87
eISSN - 1098-111X
pISSN - 0884-8173
DOI - 10.1002/int.21569
Subject(s) - maxima and minima , backpropagation , particle swarm optimization , premature convergence , artificial neural network , gradient descent , benchmark (surveying) , computer science , convergence (economics) , mathematical optimization , feedforward neural network , feed forward , local search (optimization) , algorithm , mathematics , artificial intelligence , engineering , mathematical analysis , geodesy , control engineering , economic growth , economics , geography
Traditional particle swarm optimization (PSO) has good global search ability, but it easily loses its diversity and thus leads to premature convergence. Gradient descent methods such as backpropagation (BP) algorithm have good performance in searching local minima, whereas they are apt to converge to local minima. To improve search ability, two hybrid algorithms combining two improved PSOs individually with BP are proposed to train single‐hidden‐layer feedforward neural networks in this paper. In the two improved PSOs, other than the phases of repulsion and attraction, a new phase named as a mixed phase is introduced, in which the particles are attracting and repelling simultaneously to prevent premature convergence. Moreover, a modified mutation operation is performed to help particles jump out of local minima in the improved PSOs. The proposed hybrid methods achieve better convergence performance with faster convergence rate than some commonly used PSO–BP approaches and purely global or local search methods. The experiments results on function approximation and benchmark classification problems are given to verify the effectiveness and efficiency of the proposed hybrid algorithms.