English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The state-of-the art machine learning approach to training deep neuralnetworks, backpropagation, is implausible for real neural networks: neuronsneed to know their outgoing weights; training alternates between a bottom-upforward pass (computation) and a top-down backward pass (learning); and thealgorithm often needs precise labels of many data points. Biologicallyplausible approximations to backpropagation, such as feedback alignment, solvethe weight transport problem, but not the other two. Thus, fully biologicallyplausible learning rules have so far remained elusive. Here we present a familyof learning rules that does not suffer from any of these problems. It ismotivated by the information bottleneck principle (extended with kernelmethods), in which networks learn to compress the input as much as possiblewithout sacrificing prediction of the output. The resulting rules have a3-factor Hebbian structure: they require pre- and post-synaptic firing ratesand an error signal - the third factor - consisting of a global teaching signaland a layer-specific term, both available without a top-down pass. They do notrequire precise labels; instead, they rely on the similarity between pairs ofdesired outputs. Moreover, to obtain good performance on hard problems andretain biological plausibility, our rules need divisive normalization - a knownfeature of biological networks. Finally, simulations show that our rulesperform nearly as well as backpropagation on image classification tasks.

Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks