English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Generalization of neural networks is crucial for deploying them safely in thereal world. Common training strategies to improve generalization involve theuse of data augmentations, ensembling and model averaging. In this work, wefirst establish a surprisingly simple but strong benchmark for generalizationwhich utilizes diverse augmentations within a training minibatch, and show thatthis can learn a more balanced distribution of features. Further, we proposeDiversify-Aggregate-Repeat Training (DART) strategy that first trains diversemodels using different augmentations (or domains) to explore the loss basin,and further Aggregates their weights to combine their expertise and obtainimproved generalization. We find that Repeating the step of Aggregationthroughout training improves the overall optimization trajectory and alsoensures that the individual models have a sufficiently low loss barrier toobtain improved generalization on combining them. We shed light on our approachby casting it in the framework proposed by Shen et al. and theoretically showthat it indeed generalizes better. In addition to improvements in In- Domaingeneralization, we demonstrate SOTA performance on the Domain Generalizationbenchmarks in the popular DomainBed framework as well. Our method is genericand can easily be integrated with several base training algorithms to achieveperformance gains.

DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networks