English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Understanding the global optimality in deep learning (DL) has been attractingmore and more attention recently. Conventional DL solvers, however, have notbeen developed intentionally to seek for such global optimality. In this paperwe propose a novel approximation algorithm, BPGrad, towards optimizing deepmodels globally via branch and pruning. Our BPGrad algorithm is based on theassumption of Lipschitz continuity in DL, and as a result it can adaptivelydetermine the step size for current gradient given the history of previousupdates, wherein theoretically no smaller steps can achieve the globaloptimality. We prove that, by repeating such branch-and-pruning procedure, wecan locate the global optimality within finite iterations. Empirically anefficient solver based on BPGrad for DL is proposed as well, and it outperformsconventional DL solvers such as Adagrad, Adadelta, RMSProp, and Adam in thetasks of object recognition, detection, and segmentation.

BPGrad: Towards Global Optimality in Deep Learning via Branch and Pruning