English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Stochastic gradient descent (SGD) has emerged as the quintessential method ina data scientist's toolbox. Much progress has been made in the last two decadestoward understanding the iteration complexity of SGD (in expectation andhigh-probability) in the learning theory and optimization literature. However,using SGD for high-stakes applications requires careful quantification of theassociated uncertainty. Toward that end, in this work, we establishhigh-dimensional Central Limit Theorems (CLTs) for linear functionals of onlineleast-squares SGD iterates under a Gaussian design assumption. Our main resultshows that a CLT holds even when the dimensionality is of order exponential inthe number of iterations of the online SGD, thereby enabling high-dimensionalinference with online SGD. Our proof technique involves leveraging Berry-Esseenbounds developed for martingale difference sequences and carefully evaluatingthe required moment and quadratic variation terms through recent advances inconcentration inequalities for product random matrices. We also provide anonline approach for estimating the variance appearing in the CLT (required forconstructing confidence intervals in practice) and establish consistencyresults in the high-dimensional setting.

High-dimensional Central Limit Theorems for Linear Functionals of Online Least-Squares SGD