English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A fundamental challenge in deep learning is that the optimal step sizes forupdate steps of stochastic gradient descent are unknown. In traditionaloptimization, line searches are used to determine good step sizes, however, indeep learning, it is too costly to search for good step sizes on the expectedempirical loss due to noisy losses. This empirical work shows that it ispossible to approximate the expected empirical loss on vertical cross sectionsfor common deep learning tasks considerably cheaply. This is achieved byapplying traditional one-dimensional function fitting to measured noisy lossesof such cross sections. The step to a minimum of the resulting approximation isthen used as step size for the optimization. This approach leads to a robustand straightforward optimization method which performs well across datasets andarchitectures without the need of hyperparameter tuning.

A straightforward line search approach on the expected empirical loss for stochastic deep learning problems