English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Expressivity is one of the most significant issues in assessing neuralnetworks. In this paper, we provide a quantitative analysis of the expressivityfor the deep neural network (DNN) from its dynamic model, where the Hilbertspace is employed to analyze the convergence and criticality. We study thefeature mapping of several widely used activation functions obtained by Hermitepolynomials, and find sharp declines or even saddle points in the featurespace, which stagnate the information transfer in DNNs. We then present a newactivation function design based on the Hermite polynomials for betterutilization of spatial representation. Moreover, we analyze the informationtransfer of DNNs, emphasizing the convergence problem caused by the mismatchbetween input and topological structure. We also study the effects of inputperturbations and regularization operators on critical expressivity. Ourtheoretical analysis reveals that DNNs use spatial domains for informationrepresentation and evolve to the edge of chaos as depth increases. In actualtraining, whether a particular network can ultimately arrive the edge of chaosdepends on its ability to overcome convergence and pass information to therequired network depth. Finally, we demonstrate the empirical performance ofthe proposed hypothesis via multivariate time series prediction and imageclassification examples.

The Expressivity and Training of Deep Neural Networks: toward the Edge of Chaos?