English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Visual reasoning is dominated by end-to-end neural networks scaled tobillions of model parameters and training examples. However, even the largestmodels struggle with compositional reasoning, generalization, fine-grainedspatial and temporal reasoning, and counting. Visual reasoning with largelanguage models (LLMs) as controllers can, in principle, address theselimitations by decomposing the task and solving subtasks by orchestrating a setof (visual) tools. Recently, these models achieved great performance on taskssuch as compositional visual question answering, visual grounding, and videotemporal reasoning. Nevertheless, in their current form, these models heavilyrely on human engineering of in-context examples in the prompt, which are oftendataset- and task-specific and require significant labor by highly skilledprogrammers. In this work, we present a framework that mitigates these issuesby introducing spatially and temporally abstract routines and by leveraging asmall number of labeled examples to automatically generate in-context examples,thereby avoiding human-created in-context examples. On a number of visualreasoning tasks, we show that our framework leads to consistent gains inperformance, makes LLMs as controllers setup more robust, and removes the needfor human engineering of in-context examples.

Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers