English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vision-language models like CLIP are widely used in zero-shot imageclassification due to their ability to understand various visual concepts andnatural language descriptions. However, how to fully leverage CLIP'sunprecedented human-like understanding capabilities to achieve betterperformance is still an open question. This paper draws inspiration from thehuman visual perception process: when classifying an object, humans first infercontextual attributes (e.g., background and orientation) which help separatethe foreground object from the background, and then classify the object basedon this information. Inspired by it, we observe that providing CLIP withcontextual attributes improves zero-shot image classification and mitigatesreliance on spurious features. We also observe that CLIP itself can reasonablyinfer the attributes from an image. With these observations, we propose atraining-free, two-step zero-shot classification method PerceptionCLIP. Givenan image, it first infers contextual attributes (e.g., background) and thenperforms object classification conditioning on them. Our experiments show thatPerceptionCLIP achieves better generalization, group robustness, andinterpretability. For example, PerceptionCLIP with ViT-L/14 improves the worstgroup accuracy by 16.5% on the Waterbirds dataset and by 3.5% on CelebA.

More Context, Less Distraction: Zero-shot Visual Classification by Inferring and Conditioning on Contextual Attributes