English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A central question of machine learning is how deep nets manage to learn tasksin high dimensions. An appealing hypothesis is that they achieve this feat bybuilding a representation of the data where information irrelevant to the taskis lost. For image datasets, this view is supported by the observation thatafter (and not before) training, the neural representation becomes less andless sensitive to diffeomorphisms acting on images as the signal propagatesthrough the net. This loss of sensitivity correlates with performance, andsurprisingly correlates with a gain of sensitivity to white noise acquiredduring training. These facts are unexplained, and as we demonstrate still holdwhen white noise is added to the images of the training set. Here, we (i) showempirically for various architectures that stability to image diffeomorphismsis achieved by both spatial and channel pooling, (ii) introduce a modelscale-detection task which reproduces our empirical observations on spatialpooling and (iii) compute analitically how the sensitivity to diffeomorphismsand noise scales with depth due to spatial pooling. The scalings are found todepend on the presence of strides in the net architecture. We find that theincreased sensitivity to noise is due to the perturbing noise piling up duringpooling, after being rectified by ReLU units.

How deep convolutional neural networks lose spatial information with training