English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many problems encountered in science and engineering can be formulated asestimating a low-rank object (e.g., matrices and tensors) from incomplete, andpossibly corrupted, linear measurements. Through the lens of matrix and tensorfactorization, one of the most popular approaches is to employ simple iterativealgorithms such as gradient descent (GD) to recover the low-rank factorsdirectly, which allow for small memory and computation footprints. However, theconvergence rate of GD depends linearly, and sometimes even quadratically, onthe condition number of the low-rank object, and therefore, GD slows downpainstakingly when the problem is ill-conditioned. This chapter introduces anew algorithmic approach, dubbed scaled gradient descent (ScaledGD), thatprovably converges linearly at a constant rate independent of the conditionnumber of the low-rank object, while maintaining the low per-iteration cost ofgradient descent for a variety of tasks including sensing, robust principalcomponent analysis and completion. In addition, ScaledGD continues to admitfast global convergence to the minimax-optimal solution, again almostindependent of the condition number, from a small random initialization whenthe rank is over-specified in the presence of Gaussian noise. In total,ScaledGD highlights the power of appropriate preconditioning in acceleratingnonconvex statistical estimation, where the iteration-varying preconditionerspromote desirable invariance properties of the trajectory with respect to thesymmetry in low-rank factorization without hurting generalization.

Provably Accelerating Ill-Conditioned Low-rank Estimation via Scaled Gradient Descent, Even with Overparameterization