English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we investigate the complexity of feed-forward neural networksby examining the concept of functional equivalence, which suggests thatdifferent network parameterizations can lead to the same function. We utilizethe permutation invariance property to derive a novel covering number bound forthe class of feedforward neural networks, which reveals that the complexity ofa neural network can be reduced by exploiting this property. We discuss theextensions to convolutional neural networks, residual networks, andattention-based models. We demonstrate that functional equivalence benefitsoptimization, as overparameterized networks tend to be easier to train sinceincreasing network width leads to a diminishing volume of the effectiveparameter space. Our findings offer new insights into overparameterization andhave significant implications for understanding generalization and optimizationin deep learning.

Complexity of Deep Neural Networks from the Perspective of Functional Equivalence