English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vision-and-language (VL) models with separate encoders for each modality(e.g., CLIP) have become the go-to models for zero-shot image classificationand image-text retrieval. The bulk of the evaluation of these models is,however, performed with English text only: the costly creation oflanguage-specific image-caption datasets has limited multilingual VL benchmarksto a handful of high-resource languages. In this work, we introduceBabel-ImageNet, a massively multilingual benchmark that offers (partial)translations of 1000 ImageNet labels to 92 languages, built without resortingto machine translation (MT) or requiring manual annotation. We insteadautomatically obtain reliable translations of ImageNext concepts by linkingthem -- via shared WordNet synsets -- to BabelNet, a massively multilinguallexico-semantic network. We evaluate 8 different publicly availablemultilingual CLIP models on zero-shot image classification (ZS-IC) for each ofthe 92 Babel-ImageNet languages, demonstrating a significant gap betweenEnglish ImageNet performance and that of high-resource languages (e.g., Germanor Chinese), and an even bigger gap for low-resource languages (e.g., Sinhalaor Lao). Crucially, we show that the models' ZS-IC performance onBabel-ImageNet highly correlates with their performance in image-textretrieval, validating that Babel-ImageNet is suitable for estimating thequality of the multilingual VL representation spaces for the vast majority oflanguages that lack gold image-text data. Finally, we show that the performanceof multilingual CLIP for low-resource languages can be drastically improved viacheap, parameter-efficient language-specific training. We make our code anddata publicly available: \url{https://github.com/gregor-ge/Babel-ImageNet}

Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations