English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Random feature model with a nonlinear activation function has been shown toperform asymptotically equivalent to a Gaussian model in terms of training andgeneralization errors. Analysis of the equivalent model reveals an importantyet not fully understood role played by the activation function. To addressthis issue, we study the "parameters" of the equivalent model to achieveimproved generalization performance for a given supervised learning problem. Weshow that acquired parameters from the Gaussian model enable us to define a setof optimal nonlinearities. We provide two example classes from this set, e.g.,second-order polynomial and piecewise linear functions. These functions areoptimized to improve generalization performance regardless of the actual form.We experiment with regression and classification problems, including syntheticand real (e.g., CIFAR10) data. Our numerical results validate that theoptimized nonlinearities achieve better generalization performance thanwidely-used nonlinear functions such as ReLU. Furthermore, we illustrate thatthe proposed nonlinearities also mitigate the so-called double descentphenomenon, which is known as the non-monotonic generalization performanceregarding the sample size and the model size.

Optimal Nonlinearities Improve Generalization Performance of Random Features