English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

The generalization performance of deep neural networks with regard to theoptimization algorithm is one of the major concerns in machine learning. Thisperformance can be affected by various factors. In this paper, we theoreticallyprove that the Lipschitz constant of a loss function is an important factor todiminish the generalization error of the output model obtained by Adam orAdamW. The results can be used as a guideline for choosing the loss functionwhen the optimization algorithm is Adam or AdamW. In addition, to evaluate thetheoretical bound in a practical setting, we choose the human age estimationproblem in computer vision. For assessing the generalization better, thetraining and test datasets are drawn from different distributions. Ourexperimental evaluation shows that the loss function with a lower Lipschitzconstant and maximum value improves the generalization of the model trained byAdam or AdamW.

Lipschitzness Effect of a Loss Function on Generalization Performance of Deep Neural Networks Trained by Adam and AdamW Optimizers