English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large-scale pre-trained Vision Language Models (VLMs) have proven effectivefor zero-shot classification. Despite the success, most traditional VLMs-basedmethods are restricted by the assumption of partial source supervision or idealvocabularies, which rarely satisfy the open-world scenario. In this paper, weaim at a more challenging setting, Realistic Zero-Shot Classification, whichassumes no annotation but instead a broad vocabulary. To address thischallenge, we propose the Self Structural Semantic Alignment (S^3A) framework,which extracts the structural semantic information from unlabeled data whilesimultaneously self-learning. Our S^3A framework adopts a uniqueCluster-Vote-Prompt-Realign (CVPR) algorithm, which iteratively groupsunlabeled data to derive structural semantics for pseudo-supervision. Our CVPRprocess includes iterative clustering on images, voting within each cluster toidentify initial class candidates from the vocabulary, generatingdiscriminative prompts with large language models to discern confusingcandidates, and realigning images and the vocabulary as structural semanticalignment. Finally, we propose to self-learn the CLIP image encoder with bothindividual and structural semantic alignment through a teacher-student learningstrategy. Our comprehensive experiments across various generic and fine-grainedbenchmarks demonstrate that the S^3A method offers substantial improvementsover existing VLMs-based approaches, achieving a more than 15% accuracyimprovement over CLIP on average. Our codes, models, and prompts are publiclyreleased at https://github.com/sheng-eatamath/S3A.

Towards Realistic Zero-Shot Classification via Self Structural Semantic Alignment