English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This tutorial paper surveys provably optimal alternatives to end-to-endbackpropagation (E2EBP) -- the de facto standard for training deeparchitectures. Modular training refers to strictly local training without boththe forward and the backward pass, i.e., dividing a deep architecture intoseveral nonoverlapping modules and training them separately without anyend-to-end operation. Between the fully global E2EBP and the strictly localmodular training, there are weakly modular hybrids performing training withoutthe backward pass only. These alternatives can match or surpass the performanceof E2EBP on challenging datasets such as ImageNet, and are gaining increasingattention primarily because they offer practical advantages over E2EBP, whichwill be enumerated herein. In particular, they allow for greater modularity andtransparency in deep learning workflows, aligning deep learning with themainstream computer science engineering that heavily exploits modularizationfor scalability. Modular training has also revealed novel insights aboutlearning and has further implications on other important research domains.Specifically, it induces natural and effective solutions to some importantpractical problems such as data efficiency and transferability estimation.

Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods