English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In the realm of out-of-distribution (OOD) generalization tasks, fine-tuningpre-trained models has become a prevalent strategy. Different from most priorwork that has focused on advancing learning algorithms, we systematicallyexamined how pre-trained model size, pre-training data scale, and trainingstrategies impact downstream generalization and uncertainty calibration. Weevaluated 97 models across diverse pre-trained model sizes, five pre-trainingdatasets, and five data augmentations through extensive experiments on fourdistribution shift datasets totaling over 100,000 GPU hours. Our resultsdemonstrate the significant impact of pre-trained model selection, with optimalchoices substantially improving OOD accuracy over algorithm improvement alone.We find larger models and bigger pre-training data improve OOD performance andcalibration, in contrast to some prior studies that found modern deep networksto calibrate worse than classical shallow models. Our work underscores theoverlooked importance of pre-trained model selection for out-of-distributiongeneralization and calibration.

An Empirical Investigation of Pre-trained Model Selection for Out-of-Distribution Generalization and Calibration