English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Over the past years, there has been significant interest in understanding theimplicit bias of gradient descent optimization and its connection to thegeneralization properties of overparametrized neural networks. Several worksobserved that when training linear diagonal networks on the square loss forregression tasks (which corresponds to overparametrized linear regression)gradient descent converges to special solutions, e.g., non-negative ones. Weconnect this observation to Riemannian optimization and view overparametrizedGD with identical initialization as a Riemannian GD. We use this fact forsolving non-negative least squares (NNLS), an important problem behind manytechniques, e.g., non-negative matrix factorization. We show that gradient flowon the reparametrized objective converges globally to NNLS solutions, providingconvergence rates also for its discretized counterpart. Unlike previousmethods, we do not rely on the calculation of exponential maps or geodesics. Wefurther show accelerated convergence using a second-order ODE, lending itselfto accelerated descent methods. Finally, we establish the stability againstnegative perturbations and discuss generalization to other constrainedoptimization problems.

Get rid of your constraints and reparametrize: A study in NNLS and implicit bias