English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vision-Language Models (VLMs) such as CLIP are trained on large amounts ofimage-text pairs, resulting in remarkable generalization across several datadistributions. The prohibitively expensive training and datacollection/curation costs of these models make them valuable IntellectualProperty (IP) for organizations. This motivates a vendor-client paradigm, wherea vendor trains a large-scale VLM and grants only input-output access toclients on a pay-per-query basis in a black-box setting. The client aims tominimize inference cost by distilling the VLM to a student model using thelimited available task-specific data, and further deploying this student modelin the downstream application. While naive distillation largely improves theIn-Domain (ID) accuracy of the student, it fails to transfer the superiorout-of-distribution (OOD) generalization of the VLM teacher using the limitedavailable labeled images. To mitigate this, we propose Vision-Language toVision-Align, Distill, Predict (VL2V-ADiP), which first aligns the vision andlanguage modalities of the teacher model with the vision modality of apre-trained student model, and further distills the aligned VLM embeddings tothe student. This maximally retains the pre-trained features of the student,while also incorporating the rich representations of the VLM image encoder andthe superior generalization of the text embeddings. The proposed approachachieves state-of-the-art results on the standard Domain Generalizationbenchmarks in a black-box teacher setting, and also when weights of the VLM areaccessible.

Distilling from Vision-Language Models for Improved OOD Generalization in Vision Tasks