English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Visual abductive reasoning aims to make likely explanations for visualobservations. We propose a simple yet effective Region Conditioned Adaptation,a hybrid parameter-efficient fine-tuning method that equips the frozen CLIPwith the ability to infer explanations from local visual cues. We encode``local hints'' and ``global contexts'' into visual prompts of the CLIP modelseparately at fine and coarse-grained levels. Adapters are used for fine-tuningCLIP models for downstream tasks and we design a new attention adapter, thatdirectly steers the focus of the attention map with trainable query and keyprojections of a frozen CLIP model. Finally, we train our new model with amodified contrastive loss to regress the visual feature simultaneously towardfeatures of literal description and plausible explanations. The loss enablesCLIP to maintain both perception and reasoning abilities. Experiments on theSherlock visual abductive reasoning benchmark show that the RCA significantlyoutstands previous SOTAs, ranking the \nth{1} on the leaderboards (e.g., HumanAcc: RCA 31.74 \textit{vs} CPT-CLIP 29.58, higher =better). We also validatethe RCA is generalizable to local perception benchmarks like RefCOCO. Weopen-source our project at\textit{\color{magenta}{\url{https://github.com/LUNAProject22/RPA}}}.

RCA: Region Conditioned Adaptation for Visual Abductive Reasoning