English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In Domain Generalization (DG) settings, models trained independently on agiven set of training domains have notoriously chaotic performance ondistribution shifted test domains, and stochasticity in optimization (e.g.seed) plays a big role. This makes deep learning models unreliable in realworld settings. We first show that this chaotic behavior exists even along thetraining optimization trajectory of a single model, and propose a simple modelaveraging protocol that both significantly boosts domain generalization anddiminishes the impact of stochasticity by improving the rank correlationbetween the in-domain validation accuracy and out-domain test accuracy, whichis crucial for reliable early stopping. Taking advantage of our observation, weshow that instead of ensembling unaveraged models (that is typical inpractice), ensembling moving average models (EoA) from independent runs furtherboosts performance. We theoretically explain the boost in performance ofensembling and model averaging by adapting the well known Bias-Variancetrade-off to the domain generalization setting. On the DomainBed benchmark,when using a pre-trained ResNet-50, this ensemble of averages achieves anaverage of $68.0\%$, beating vanilla ERM (w/o averaging/ensembling) by $\sim4\%$, and when using a pre-trained RegNetY-16GF, achieves an average of$76.6\%$, beating vanilla ERM by $6\%$. Our code is available athttps://github.com/salesforce/ensemble-of-averages.

Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization