English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we provide a geometric interpretation of the structure ofshallow neural networks characterized by one hidden layer, a ramp activationfunction, an ${\mathcal L}^2$ Schatten class (or Hilbert-Schmidt) costfunction, input space ${\mathbb R}^M$, output space ${\mathbb R}^Q$ with $Q\leqM$, and training input sample size $N>QM$. We prove an upper bound on theminimum of the cost function of order $O(\delta_P$ where $\delta_P$ measuresthe signal to noise ratio of training inputs. We obtain an approximateoptimizer using projections adapted to the averages $\overline{x_{0,j}}$ oftraining input vectors belonging to the same output vector $y_j$,$j=1,\dots,Q$. In the special case $M=Q$, we explicitly determine an exactdegenerate local minimum of the cost function; the sharp value differs from theupper bound obtained for $Q\leq M$ by a relative error $O(\delta_P^2)$. Theproof of the upper bound yields a constructively trained network; we show thatit metrizes the $Q$-dimensional subspace in the input space ${\mathbb R}^M$spanned by $\overline{x_{0,j}}$, $j=1,\dots,Q$. We comment on thecharacterization of the global minimum of the cost function in the givencontext.

Geometric structure of shallow neural networks and constructive ${\mathcal L}^2$ cost minimization