English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the binary and continuous negative-margin perceptrons as simplenon-convex neural network models learning random rules and associations. Weanalyze the geometry of the landscape of solutions in both models and findimportant similarities and differences. Both models exhibit subdominantminimizers which are extremely flat and wide. These minimizers coexist with abackground of dominant solutions which are composed by an exponential number ofalgorithmically inaccessible small clusters for the binary case (the frozen1-RSB phase) or a hierarchical structure of clusters of different sizes for thespherical case (the full RSB phase). In both cases, when a certain threshold inconstraint density is crossed, the local entropy of the wide flat minimabecomes non-monotonic, indicating a break-up of the space of robust solutionsinto disconnected components. This has a strong impact on the behavior ofalgorithms in binary models, which cannot access the remaining isolatedclusters. For the spherical case the behaviour is different, since even beyondthe disappearance of the wide flat minima the remaining solutions are shown toalways be surrounded by a large number of other solutions at any distance, upto capacity. Indeed, we exhibit numerical evidence that algorithms seem to findsolutions up to the SAT/UNSAT transition, that we compute here using an 1RSBapproximation. For both models, the generalization performance as a learningdevice is shown to be greatly improved by the existence of wide flat minimizerseven when trained in the highly underconstrained regime of very negativemargins.

Typical and atypical solutions in non-convex neural networks with discrete and continuous weights