English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The success of deep learning over the past decade mainly relies ongradient-based optimisation and backpropagation. This paper focuses onanalysing the performance of first-order gradient-based optimisationalgorithms, gradient descent and proximal gradient, with time-varyingnon-convex cost function under (proximal) Polyak-{\L}ojasiewicz condition.Specifically, we focus on using the forward mode of automatic differentiationto compute gradients in the fast-changing problems where calculating gradientsusing the backpropagation algorithm is either impossible or inefficient. Upperbounds for tracking and asymptotic errors are derived for various cases,showing the linear convergence to a solution or a neighbourhood of an optimalsolution, where the convergence rate decreases with the increase in thedimension of the problem. We show that for a solver with constraints oncomputing resources, the number of forward gradient iterations at each step canbe a design parameter that trades off between the tracking performance andcomputing constraints.

First order online optimisation using forward gradients in over-parameterised systems