English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large pre-trained vision and language models have demonstrated remarkablecapacities for various tasks. However, solving the knowledge-based visualreasoning tasks remains challenging, which requires a model to comprehensivelyunderstand image content, connect the external world knowledge, and performstep-by-step reasoning to answer the questions correctly. To this end, wepropose a novel framework named Interactive Prompting Visual Reasoner (IPVR)for few-shot knowledge-based visual reasoning. IPVR contains three stages, see,think and confirm. The see stage scans the image and grounds the visual conceptcandidates with a visual perception model. The think stage adopts a pre-trainedlarge language model (LLM) to attend to the key concepts from candidatesadaptively. It then transforms them into text context for prompting with avisual captioning model and adopts the LLM to generate the answer. The confirmstage further uses the LLM to generate the supporting rationale to the answer,verify the generated rationale with a cross-modality classifier and ensure thatthe rationale can infer the predicted output consistently. We conductexperiments on a range of knowledge-based visual reasoning datasets. We foundour IPVR enjoys several benefits, 1). it achieves better performance than theprevious few-shot learning baselines; 2). it enjoys the total transparency andtrustworthiness of the whole reasoning process by providing rationales for eachreasoning step; 3). it is computation-efficient compared with other fine-tuningbaselines.

See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoning