English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we provide a theoretical analysis of the inductive biases inconvolutional neural networks (CNNs). We start by examining the universality ofCNNs, i.e., the ability to approximate any continuous functions. We prove thata depth of $\mathcal{O}(\log d)$ suffices for deep CNNs to achieve thisuniversality, where $d$ in the input dimension. Additionally, we establish thatlearning sparse functions with CNNs requires only$\widetilde{\mathcal{O}}(\log^2d)$ samples, indicating that deep CNNs canefficiently capture {\em long-range} sparse correlations. These results aremade possible through a novel combination of the multichanneling anddownsampling when increasing the network depth. We also delve into the distinctroles of weight sharing and locality in CNNs. To this end, we compare theperformance of CNNs, locally-connected networks (LCNs), and fully-connectednetworks (FCNs) on a simple regression task, where LCNs can be viewed as CNNswithout weight sharing. On the one hand, we prove that LCNs require${\Omega}(d)$ samples while CNNs need only $\widetilde{\mathcal{O}}(\log^2d)$samples, highlighting the critical role of weight sharing. On the other hand,we prove that FCNs require $\Omega(d^2)$ samples, whereas LCNs need only$\widetilde{\mathcal{O}}(d)$ samples, underscoring the importance of locality.These provable separations quantify the difference between the two biases, andthe major observation behind our proof is that weight sharing and localitybreak different symmetries in the learning process.

Theoretical Analysis of Inductive Biases in Deep Convolutional Networks