English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study mechanisms to characterize how the asymptotic convergence ofbackpropagation in deep architectures, in general, is related to the networkstructure, and how it may be influenced by other design choices includingactivation type, denoising and dropout rate. We seek to analyze whether networkarchitecture and input data statistics may guide the choices of learningparameters and vice versa. Given the broad applicability of deep architectures,this issue is interesting both from theoretical and a practical standpoint.Using properties of general nonconvex objectives (with first-orderinformation), we first build the association between structural, distributionaland learnability aspects of the network vis-\`a-vis their interaction withparameter convergence rates. We identify a nice relationship between featuredenoising and dropout, and construct families of networks that achieve the samelevel of convergence. We then derive a workflow that provides systematicguidance regarding the choice of network sizes and learning parameters oftenmediated4 by input statistics. Our technical results are corroborated by anextensive set of evaluations, presented in this paper as well as independentempirical observations reported by other groups. We also perform experimentsshowing the practical implications of our framework for choosing the bestfully-connected design for a given problem.

On architectural choices in deep learning: From network structure to gradient convergence and parameter estimation