English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Large-scale vision-language models (VLMs), e.g., CLIP, learn broad visualconcepts from tedious training data, showing superb generalization ability.Amount of prompt learning methods have been proposed to efficiently adapt theVLMs to downstream tasks with only a few training samples. We introduce a novelmethod to improve the prompt learning of vision-language models byincorporating pre-trained large language models (LLMs), called Dual-AlignedPrompt Tuning (DuAl-PT). Learnable prompts, like CoOp, implicitly model thecontext through end-to-end training, which are difficult to control andinterpret. While explicit context descriptions generated by LLMs, like GPT-3,can be directly used for zero-shot classification, such prompts are overlyrelying on LLMs and still underexplored in few-shot domains. With DuAl-PT, wepropose to learn more context-aware prompts, benefiting from both explicit andimplicit context modeling. To achieve this, we introduce a pre-trained LLM togenerate context descriptions, and we encourage the prompts to learn from theLLM's knowledge by alignment, as well as the alignment between prompts andlocal image features. Empirically, DuAl-PT achieves superior performance on 11downstream datasets on few-shot recognition and base-to-new generalization.Hopefully, DuAl-PT can serve as a strong baseline. Code will be available.

Context-Aware Prompt Tuning for Vision-Language Model with Dual-Alignment