A Simplified Natural Gradient LearningAlgorithm

Adaptive natural gradient learning avoids singularities in the parameterspace of multilayer perceptrons. However, it requires a larger numberof additional parameters than ordinary backpropagation in the form ofthe Fisher information matrix. This paper describes a new approach tonatural gradient learning that uses a smaller Fisher information matrix.It also uses a prior distribution on the neural network parameters and anannealed learning rate. While this new approach is computationally simpler,its performance is comparable to that of adaptive natural gradientlearning