English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Recent years have seen many insights on deep learning optimisation beingbrought forward by finding implicit regularisation effects of commonly usedgradient-based optimisers. Understanding implicit regularisation can not onlyshed light on optimisation dynamics, but it can also be used to improveperformance and stability across problem domains, from supervised learning totwo-player games such as Generative Adversarial Networks. An avenue for findingsuch implicit regularisation effects has been quantifying the discretisationerrors of discrete optimisers via continuous-time flows constructed by backwarderror analysis (BEA). The current usage of BEA is not without limitations,since not all the vector fields of continuous-time flows obtained using BEA canbe written as a gradient, hindering the construction of modified lossesrevealing implicit regularisers. In this work, we provide a novel approach touse BEA, and show how our approach can be used to construct continuous-timeflows with vector fields that can be written as gradients. We then use this tofind previously unknown implicit regularisation effects, such as those inducedby multiple stochastic gradient descent steps while accounting for the exactdata batches used in the updates, and in generally differentiable two-playergames.

Implicit regularisation in stochastic gradient descent: from single-objective to two-player games