English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Learning long-term dependencies is a key long-standing challenge of recurrentneural networks (RNNs). Hierarchical recurrent neural networks (HRNNs) havebeen considered a promising approach as long-term dependencies are resolvedthrough shortcuts up and down the hierarchy. Yet, the memory requirements ofTruncated Backpropagation Through Time (TBPTT) still prevent training them onvery long sequences. In this paper, we empirically show that in (deep) HRNNs,propagating gradients back from higher to lower levels can be replaced bylocally computable losses, without harming the learning capability of thenetwork, over a wide range of tasks. This decoupling by local losses reducesthe memory requirements of training by a factor exponential in the depth of thehierarchy in comparison to standard TBPTT.

Decoupling Hierarchical Recurrent Neural Networks With Locally Computable Losses