English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Equilibrium Propagation (EP) is a biologically-inspired algorithm forconvergent RNNs with a local learning rule that comes with strong theoreticalguarantees. The parameter updates of the neural network during the creditassignment phase have been shown mathematically to approach the gradientsprovided by Backpropagation Through Time (BPTT) when the network isinfinitesimally nudged toward its target. In practice, however, training anetwork with the gradient estimates provided by EP does not scale to visualtasks harder than MNIST. In this work, we show that a bias in the gradientestimate of EP, inherent in the use of finite nudging, is responsible for thisphenomenon and that cancelling it allows training deep ConvNets by EP. We showthat this bias can be greatly reduced by using symmetric nudging (a positivenudging and a negative one). We also generalize previous EP equations to thecase of cross-entropy loss (by opposition to squared error). As a result ofthese advances, we are able to achieve a test error of 11.7% on CIFAR-10 by EP,which approaches the one achieved by BPTT and provides a major improvement withrespect to the standard EP approach with same-sign nudging that gives 86% testerror. We also apply these techniques to train an architecture with asymmetricforward and backward connections, yielding a 13.2% test error. These resultshighlight EP as a compelling biologically-plausible approach to compute errorgradients in deep neural networks.

Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias