English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Finding best architectures of learning machines, such as deep neuralnetworks, is a well-known technical and theoretical challenge. Recent work byMellor et al (2021) showed that there may exist correlations between theaccuracies of trained networks and the values of some easily computablemeasures defined on randomly initialised networks which may enable to searchtens of thousands of neural architectures without training. Mellor et al usedthe Hamming distance evaluated over all ReLU neurons as such a measure.Motivated by these findings, in our work, we ask the question of the existenceof other and perhaps more principled measures which could be used asdeterminants of success of a given neural architecture. In particular, weexamine, if the dimensionality and quasi-orthogonality of neural networks'feature space could be correlated with the network's performance aftertraining. We showed, using the setup as in Mellor et al, that dimensionalityand quasi-orthogonality may jointly serve as network's performancediscriminants. In addition to offering new opportunities to accelerate neuralarchitecture search, our findings suggest important relationships between thenetworks' final performance and properties of their randomly initialisedfeature spaces: data dimension and quasi-orthogonality.

Quasi-orthogonality and intrinsic dimensions as measures of learning and generalisation