English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recently, there has been significant progress in understanding theconvergence and generalization properties of gradient-based methods fortraining overparameterized learning models. However, many aspects including therole of small random initialization and how the various parameters of the modelare coupled during gradient-based updates to facilitate good generalizationremain largely mysterious. A series of recent papers have begun to study thisrole for non-convex formulations of symmetric Positive Semi-Definite (PSD)matrix sensing problems which involve reconstructing a low-rank PSD matrix froma few linear measurements. The underlying symmetry/PSDness is crucial toexisting convergence and generalization guarantees for this problem. In thispaper, we study a general overparameterized low-rank matrix sensing problemwhere one wishes to reconstruct an asymmetric rectangular low-rank matrix froma few linear measurements. We prove that an overparameterized model trained viafactorized gradient descent converges to the low-rank matrix generating themeasurements. We show that in this setting, factorized gradient descent enjoystwo implicit properties: (1) coupling of the trajectory of gradient descentwhere the factors are coupled in various ways throughout the gradient updatetrajectory and (2) an algorithmic regularization property where the iteratesshow a propensity towards low-rank models despite the overparameterized natureof the factorized model. These two implicit properties in turn allow us to showthat the gradient descent trajectory from small random initialization movestowards solutions that are both globally optimal and generalize well.

Implicit Balancing and Regularization: Generalization and Convergence Guarantees for Overparameterized Asymmetric Matrix Sensing