English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

DenseNets introduce concatenation-type skip connections that achievestate-of-the-art accuracy in several computer vision tasks. In this paper, wereveal that the topology of the concatenation-type skip connections is closelyrelated to the gradient propagation which, in turn, enables a predictablebehavior of DNNs' test performance. To this end, we introduce a new metriccalled NN-Mass to quantify how effectively information flows through DNNs.Moreover, we empirically show that NN-Mass also works for other types of skipconnections, e.g., for ResNets, Wide-ResNets (WRNs), and MobileNets, whichcontain addition-type skip connections (i.e., residuals or inverted residuals).As such, for both DenseNet-like CNNs and ResNets/WRNs/MobileNets, ourtheoretically grounded NN-Mass can identify models with similar accuracy,despite having significantly different size/compute requirements. Detailedexperiments on both synthetic and real datasets (e.g., MNIST, CIFAR-10,CIFAR-100, ImageNet) provide extensive evidence for our insights. Finally, theclosed-form equation of our NN-Mass enables us to design significantlycompressed DenseNets (for CIFAR-10) and MobileNets (for ImageNet) directly atinitialization without time-consuming training and/or searching.

How does topology influence gradient propagation and model performance of deep networks with DenseNet-type skip connections?