English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Modular neural networks without additional training have recently been shownto surpass end-to-end neural networks on challenging vision-language tasks. Thelatest such methods simultaneously introduce LLM-based code generation to buildprograms and a number of skill-specific, task-oriented modules to execute them.In this paper, we focus on ViperGPT and ask where its additional performancecomes from and how much is due to the (state-of-art, end-to-end) BLIP-2 modelit subsumes vs. additional symbolic components. To do so, we conduct acontrolled study (comparing end-to-end, modular, and prompting-based methodsacross several VQA benchmarks). We find that ViperGPT's reported gains overBLIP-2 can be attributed to its selection of task-specific modules, and when werun ViperGPT using a more task-agnostic selection of modules, these gains goaway. Additionally, ViperGPT retains much of its performance if we makeprominent alterations to its selection of modules: e.g. removing or retainingonly BLIP-2. Finally, we compare ViperGPT against a prompting-baseddecomposition strategy and find that, on some benchmarks, modular approachessignificantly benefit by representing subtasks with natural language, insteadof code.

Analyzing Modular Approaches for Visual Question Decomposition