English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Contrastively trained text-image models have the remarkable ability toperform zero-shot classification, that is, classifying previously unseen imagesinto categories that the model has never been explicitly trained to identify.However, these zero-shot classifiers need prompt engineering to achieve highaccuracy. Prompt engineering typically requires hand-crafting a set of promptsfor individual downstream tasks. In this work, we aim to automate this promptengineering and improve zero-shot accuracy through prompt ensembling. Inparticular, we ask "Given a large pool of prompts, can we automatically scorethe prompts and ensemble those that are most suitable for a particulardownstream dataset, without needing access to labeled validation data?". Wedemonstrate that this is possible. In doing so, we identify several pathologiesin a naive prompt scoring method where the score can be easily overconfidentdue to biases in pre-training and test data, and we propose a novel promptscoring method that corrects for the biases. Using our proposed scoring methodto create a weighted average prompt ensemble, our method outperforms equalaverage ensemble, as well as hand-crafted prompts, on ImageNet, 4 of itsvariants, and 11 fine-grained classification benchmarks, all while being fullyautomatic, optimization-free, and not requiring access to labeled validationdata.

A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Models