English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many optimizers have been proposed for training deep neural networks, andthey often have multiple hyperparameters, which make it tricky to benchmarktheir performance. In this work, we propose a new benchmarking protocol toevaluate both end-to-end efficiency (training a model from scratch withoutknowing the best hyperparameter) and data-addition training efficiency (thepreviously selected hyperparameters are used for periodically re-training themodel with newly collected data). For end-to-end efficiency, unlike previouswork that assumes random hyperparameter tuning, which over-emphasizes thetuning time, we propose to evaluate with a bandit hyperparameter tuningstrategy. A human study is conducted to show that our evaluation protocolmatches human tuning behavior better than the random search. For data-additiontraining, we propose a new protocol for assessing the hyperparametersensitivity to data shift. We then apply the proposed benchmarking framework to7 optimizers and various tasks, including computer vision, natural languageprocessing, reinforcement learning, and graph mining. Our results show thatthere is no clear winner across all the tasks.

How much progress have we made in neural network training? A New Evaluation Protocol for Benchmarking Optimizers