Opening the Black Box: the Relationship between Neural Networks and Linear Discriminant Functions

Over the last ten years feed-forward neural networks have become a popular tool for statistical decision making. During this time, they have been applied in many fields, including cytological classification. Neural networks are often treated as a black box, whose inner workings are concealed from the researcher. This is unfortunate, since the inner workings of a neural network can be understood in a manner similar to that of a linear discriminant function, which is the standard tool that researchers use for decision making. This paper discusses feed-forward neural networks and some methods to improve their performance for classification problems. Their relationship to discriminant functions will be examined for a simple two-dimensional classification problem.