English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite significant recent advances in deep neural networks, training themremains a challenge due to the highly non-convex nature of the objectivefunction. State-of-the-art methods rely on error backpropagation, which suffersfrom several well-known issues, such as vanishing and exploding gradients,inability to handle non-differentiable nonlinearities and to parallelizeweight-updates across layers, and biological implausibility. These limitationscontinue to motivate exploration of alternative training algorithms, includingseveral recently proposed auxiliary-variable methods which break the complexnested objective function into local subproblems. However, those techniques aremainly offline (batch), which limits their applicability to extremely largedatasets, as well as to online, continual or reinforcement learning. The maincontribution of our work is a novel online (stochastic/mini-batch) alternatingminimization (AM) approach for training deep neural networks, together with thefirst theoretical convergence guarantees for AM in stochastic settings andpromising empirical results on a variety of architectures and datasets.

Beyond Backprop: Online Alternating Minimization with Auxiliary Variables