English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Web-crawled datasets have enabled remarkable generalization capabilities inrecent image-text models such as CLIP (Contrastive Language-Image pre-training)or Flamingo, but little is known about the dataset creation processes. In thiswork, we introduce a testbed of six publicly available data sources - YFCC,LAION, Conceptual Captions, WIT, RedCaps, Shutterstock - to investigate howpre-training distributions induce robustness in CLIP. We find that theperformance of the pre-training data varies substantially across distributionshifts, with no single data source dominating. Moreover, we systematicallystudy the interactions between these data sources and find that combiningmultiple sources does not necessarily yield better models, but rather dilutesthe robustness of the best individual data source. We complement our empiricalfindings with theoretical insights from a simple setting, where combining thetraining data also results in diluted robustness. In addition, our theoreticalmodel provides a candidate explanation for the success of the CLIP-based datafiltering technique recently employed in the LAION dataset. Overall our resultsdemonstrate that simply gathering a large amount of data from the web is notthe most effective way to build a pre-training dataset for robustgeneralization, necessitating further study into dataset design. Code isavailable at https://github.com/mlfoundations/clip_quality_not_quantity.

Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP