English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Understanding the fundamental mechanism behind the success of deep neuralnetworks is one of the key challenges in the modern machine learningliterature. Despite numerous attempts, a solid theoretical analysis is yet tobe developed. In this paper, we develop a novel unified framework to reveal ahidden regularization mechanism through the lens of convex optimization. Wefirst show that the training of multiple three-layer ReLU sub-networks withweight decay regularization can be equivalently cast as a convex optimizationproblem in a higher dimensional space, where sparsity is enforced via a group$\ell_1$-norm regularization. Consequently, ReLU networks can be interpreted ashigh dimensional feature selection methods. More importantly, we then provethat the equivalent convex problem can be globally optimized by a standardconvex optimization solver with a polynomial-time complexity with respect tothe number of samples and data dimension when the width of the network isfixed. Finally, we numerically validate our theoretical results via experimentsinvolving both synthetic and real datasets.

Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs