Higher-order probabilistic perceptrons as Bayesian inference engines

An explicit structural connection is established between the Bayes optimal classifier operating on K binary input variables and a corresponding two-layer perceptron having normalized output activities and couplings from input to output units of all orders up to K. With suitable modification of connection weights and biases, such a higher-order probabilistic perceptron should in principle be able to learn the statistics of the classification problem and match the a posteriori probabilities given by Bayes optimal inference. Specific training algorithms are developed that allow this goal to be approximated in a controlled variational sense. An application to the task of discriminating between stable and unstable nuclides in nuclear physics yields network models with predictive performance comparable to the best that has been achieved with conventional multilayer perceptrons containing only pairwise connections.