English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We present a framework for analyzing the exact dynamics of a class of onlinelearning algorithms in the high-dimensional scaling limit. Our results areapplied to two concrete examples: online regularized linear regression andprincipal component analysis. As the ambient dimension tends to infinity, andwith proper time scaling, we show that the time-varying joint empiricalmeasures of the target feature vector and its estimates provided by thealgorithms will converge weakly to a deterministic measured-valued process thatcan be characterized as the unique solution of a nonlinear PDE. Numericalsolutions of this PDE can be efficiently obtained. These solutions lead toprecise predictions of the performance of the algorithms, as many practicalperformance metrics are linear functionals of the joint empirical measures. Inaddition to characterizing the dynamic performance of online learningalgorithms, our asymptotic analysis also provides useful insights. Inparticular, in the high-dimensional limit, and due to exchangeability, theoriginal coupled dynamics associated with the algorithms will be asymptotically"decoupled", with each coordinate independently solving a 1-D effectiveminimization problem via stochastic gradient descent. Exploiting this insightfor nonconvex optimization problems may prove an interesting line of futureresearch.

Scaling Limit: Exact and Tractable Analysis of Online Learning Algorithms with Applications to Regularized Regression and PCA