English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We investigate the expressive power of depth-2 bandlimited random neuralnetworks. A random net is a neural network where the hidden layer parametersare frozen with random assignment, and only the output layer parameters aretrained by loss minimization. Using random weights for a hidden layer is aneffective method to avoid non-convex optimization in standard gradient descentlearning. It has also been adopted in recent deep learning theories. Despitethe well-known fact that a neural network is a universal approximator, in thisstudy, we mathematically show that when hidden parameters are distributed in abounded domain, the network may not achieve zero approximation error. Inparticular, we derive a new nontrivial approximation error lower bound. Theproof utilizes the technique of ridgelet analysis, a harmonic analysis methoddesigned for neural networks. This method is inspired by fundamental principlesin classical signal processing, specifically the idea that signals with limitedbandwidth may not always be able to perfectly recreate the original signal. Wecorroborate our theoretical results with various simulation studies, andgenerally, two main take-home messages are offered: (i) Not any distributionfor selecting random weights is feasible to build a universal approximator;(ii) A suitable assignment of random weights exists but to some degree isassociated with the complexity of the target function.

How Powerful are Shallow Neural Networks with Bandlimited Random Weights?