Premium
Are multi‐layer backpropagation networks catastrophically amnesic?
Author(s) -
YAMAGUCHI MAKOTO
Publication year - 2004
Publication title -
scandinavian journal of psychology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.743
H-Index - 72
eISSN - 1467-9450
pISSN - 0036-5564
DOI - 10.1111/j.1467-9450.2004.00417.x
Subject(s) - forgetting , backpropagation , set (abstract data type) , connectionism , interference (communication) , interference theory , artificial neural network , artificial intelligence , computer science , psychology , pattern recognition (psychology) , cognitive psychology , cognition , working memory , channel (broadcasting) , telecommunications , neuroscience , programming language
Connectionist models with a backpropagation learning rule are known to have a serious problem. Such models exhibit catastrophic interference (or forgetting) with sequential training. Having learned a set of patterns, if the model is trained on another set of patterns, its performance on the first set can dramatically deteriorate very rapidly. The present study reconsiders this issue with four simulations. The model learned arithmetic facts sequentially, but the interference was only modest with random (hence approximately orthogonal) inputs. Essentially the same result was obtained when the inputs are made less orthogonal by adding irrelevant elements. Reducing the number of hidden units did not have major effects. This study suggests that the interference problem has been somewhat overstated.