English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Out-of-distribution (OOD) generalization is a complicated problem due to theidiosyncrasies of possible distribution shifts between training and testdomains. Most benchmarks employ diverse datasets to address this issue;however, the degree of the distribution shift between the training domains andthe test domains of each dataset remains largely fixed. This may lead to biasedconclusions that either underestimate or overestimate the actual OODperformance of a model. Our study delves into a more nuanced evaluation settingthat covers a broad range of shift degrees. We show that the robustness ofmodels can be quite brittle and inconsistent under different degrees ofdistribution shifts, and therefore one should be more cautious when drawingconclusions from evaluations under a limited range of degrees. In addition, weobserve that large-scale pre-trained models, such as CLIP, are sensitive toeven minute distribution shifts of novel downstream tasks. This indicates thatwhile pre-trained representations may help improve downstream in-distributionperformance, they could have minimal or even adverse effects on generalizationin certain OOD scenarios of the downstream task if not used properly. In lightof these findings, we encourage future research to conduct evaluations across abroader range of shift degrees whenever possible.

Robustness May be More Brittle than We Think under Different Degrees of Distribution Shifts