Symbol processing — a challenge for neural networks?

In recent years neural networks have often been proposed as a ‘miracle solution’ which can be widely applied, both to build models of human cognition and to construct intelligent systems for industry. Some proponents of neural networks (often termed ‘connectionists’) have even gone so far as to suggest that their models can supplant those of ‘classical’ AI (as exemplified by knowledge-based systems, often termed ‘symbolic AI’). There is a fierce debate as to which of these two schools of thought actually has the most powerful computational model. Proponents of symbolic AI often level the criticism at connectionists that neural networks are inadequate for performing the processing of symbols and symbol structures. Such processing is observed in human cognition and is necessary for the performance of many commercially important tasks. Symbol processing by neural networks is such a hot topic that a future special issue of this journal will be wholly dedicated to connectionist models of symbol processing. However, the motivation behind this paper is to focus on someof the most important criticisms levelled at connectionism by proponents of symbolic AI and to provide a brief introduction to this ongoing debate. It is impossible to give a complete overview of this debate in a paper of this length; readers wanting more information are referred to the further reading suggested at the end. One criticism of connectionist systems is that they do not possessstructural systematicity . Structural systematicity can be loosely defined as the ability to process new items which have a complexity greater than those items