English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The implicit bias of neural networks has been extensively studied in recentyears. Lyu and Li [2019] showed that in homogeneous networks trained with theexponential or the logistic loss, gradient flow converges to a KKT point of themax margin problem in the parameter space. However, that leaves open thequestion of whether this point will generally be an actual optimum of the maxmargin problem. In this paper, we study this question in detail, for severalneural network architectures involving linear and ReLU activations. Perhapssurprisingly, we show that in many cases, the KKT point is not even a localoptimum of the max margin problem. On the flip side, we identify multiplesettings where a local or global optimum can be guaranteed.

On Margin Maximization in Linear and ReLU Networks