Premium
Unsupervised Discovery of Nonlinear Structure Using Contrastive Backpropagation
Author(s) -
Hinton Geoffrey,
Osindero Simon,
Welling Max,
Teh YeeWhye
Publication year - 2006
Publication title -
cognitive science
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.498
H-Index - 114
eISSN - 1551-6709
pISSN - 0364-0213
DOI - 10.1207/s15516709cog0000_76
Subject(s) - artificial neural network , backpropagation , energy (signal processing) , nonlinear system , computer science , artificial intelligence , pattern recognition (psychology) , unit vector , mathematics , physics , statistics , mathematical analysis , quantum mechanics
We describe a way of modeling high‐dimensional data vectors by using an unsupervised, nonlinear, multilayer neural network in which the activity of each neuron‐like unit makes an additive contribution to a global energy score that indicates how surprised the network is by the data vector. The connection weights that determine how the activity of each unit depends on the activities in earlier layers are learned by minimizing the energy assigned to data vectors that are actually observed and maximizing the energy assigned to “confabulations” that are generated by perturbing an observed data vector in a direction that decreases its energy under the current model.