English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Equilibrium Propagation (EP) is a biologically inspired learning algorithmfor convergent recurrent neural networks, i.e. RNNs that are fed by a staticinput x and settle to a steady state. Training convergent RNNs consists inadjusting the weights until the steady state of output neurons coincides with atarget y. Convergent RNNs can also be trained with the more conventionalBackpropagation Through Time (BPTT) algorithm. In its original formulation EPwas described in the case of real-time neuronal dynamics, which iscomputationally costly. In this work, we introduce a discrete-time version ofEP with simplified equations and with reduced simulation time, bringing EPcloser to practical machine learning tasks. We first prove theoretically, aswell as numerically that the neural and weight updates of EP, computed byforward-time dynamics, are step-by-step equal to the ones obtained by BPTT,with gradients computed backward in time. The equality is strict when thetransition function of the dynamics derives from a primitive function and thesteady state is maintained long enough. We then show for more standarddiscrete-time neural network dynamics that the same property is approximatelyrespected and we subsequently demonstrate training with EP with equivalentperformance to BPTT. In particular, we define the first convolutionalarchitecture trained with EP achieving ~ 1% test error on MNIST, which is thelowest error reported with EP. These results can guide the development of deepneural networks trained with EP.

Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input