English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Generative training has been demonstrated to be powerful for buildingvisual-language models. However, on zero-shot discriminative benchmarks, thereis still a performance gap between models trained with generative anddiscriminative objectives. In this paper, we aim to narrow this gap byimproving the efficacy of generative training on classification tasks, withoutany finetuning processes or additional modules.  Specifically, we focus on narrowing the gap between the generative captionerand the CLIP classifier. We begin by analysing the predictions made by thecaptioner and classifier and observe that the caption generation inherits thedistribution bias from the language model trained with pure text modality,making it less grounded on the visual signal. To tackle this problem, weredesign the scoring objective for the captioner to alleviate thedistributional bias and focus on measuring the gain of information brought bythe visual inputs. We further design a generative training objective to matchthe evaluation objective. We name our model trained and evaluated from thenovel procedures as Information Gain (IG) captioner. We pretrain the models onthe public Laion-5B dataset and perform a series of discriminative evaluations.For the zero-shot classification on ImageNet, IG captioner achieves $> 18\%$improvements over the standard captioner, achieving comparable performanceswith the CLIP classifier. IG captioner also demonstrated strong performance onzero-shot image-text retrieval tasks on MSCOCO and Flickr30K. We hope thispaper inspires further research towards unifying generative and discriminativetraining procedures for visual-language models.

IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers