English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Modern supervised semantic segmentation methods are usually finetuned basedon the supervised or self-supervised models pre-trained on ImageNet. Recentwork shows that transferring the knowledge from CLIP to semantic segmentationvia prompt learning can achieve promising performance. The performance boostcomes from the feature enhancement with multimodal alignment, i.e., the dotproduct between vision and text embeddings. However, how to improve themultimodal alignment for better transfer performance in dense tasks remainsunderexplored. In this work, we focus on improving the quality of vision-textalignment from two aspects of prompting design and loss function, and presentan instance-conditioned prompting with contrastive learning (ICPC) framework.First, compared with the static prompt designs, we reveal that dynamicprompting conditioned on image content can more efficiently utilize the textencoder for complex dense tasks. Second, we propose an align-guided contrastiveloss to refine the alignment of vision and text embeddings. We further proposelightweight multi-scale alignment for better performance. Extensive experimentson three large-scale datasets (ADE20K, COCO-Stuff10k, and ADE20K-Full)demonstrate that ICPC brings consistent improvements across diverse backbones.Taking ResNet-50 as an example, ICPC outperforms the state-of-the-artcounterpart by 1.71%, 1.05%, and 1.41% mIoU on the three datasets,respectively.

ICPC: Instance-Conditioned Prompting with Contrastive Learning for Semantic Segmentation