English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Foundation models like CLIP are trained on hundreds of millions of samplesand effortlessly generalize to new tasks and inputs. Out of the box, CLIP showsstellar zero-shot and few-shot capabilities on a wide range ofout-of-distribution (OOD) benchmarks, which prior works attribute mainly totoday's large and comprehensive training dataset (like LAION). However, it isquestionable how meaningful terms like out-of-distribution generalization arefor CLIP as it seems likely that web-scale datasets like LAION simply containmany samples that are similar to common OOD benchmarks originally designed forImageNet. To test this hypothesis, we retrain CLIP on pruned LAION splits thatreplicate ImageNet's train-test similarity with respect to common OODbenchmarks. While we observe a performance drop on some benchmarks,surprisingly, CLIP's overall performance remains high. This shows that hightrain-test similarity is insufficient to explain CLIP's OOD performance, andother properties of the training data must drive CLIP to learn moregeneralizable representations. Additionally, by pruning data points that aredissimilar to the OOD benchmarks, we uncover a 100M split of LAION($\frac{1}{4}$th of its original size) on which CLIP can be trained to matchits original OOD performance.

Does CLIP's Generalization Performance Mainly Stem from High Train-Test Similarity?