English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We study the loss landscape of two-layer mildly overparameterized ReLU neuralnetworks on a generic finite input dataset for the squared error loss. Ourapproach involves bounding the dimension of the sets of local and global minimausing the rank of the Jacobian of the parameterization map. Using results onrandom binary matrices, we show most activation patterns correspond toparameter regions with no bad differentiable local minima. Furthermore, forone-dimensional input data, we show most activation regions realizable by thenetwork contain a high dimensional set of global minima and no bad localminima. We experimentally confirm these results by finding a phase transitionfrom most regions having full rank to many regions having deficient rankdepending on the amount of overparameterization.

Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape