English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large language models (LLMs) have demonstrated excellent zero-shotgeneralization to new language tasks. However, effective utilization of LLMsfor zero-shot visual question-answering (VQA) remains challenging, primarilydue to the modality disconnection and task disconnection between LLM and VQAtask. End-to-end training on vision and language data may bridge thedisconnections, but is inflexible and computationally expensive. To addressthis issue, we propose \emph{Img2Prompt}, a plug-and-play module that providesthe prompts that can bridge the aforementioned modality and taskdisconnections, so that LLMs can perform zero-shot VQA tasks without end-to-endtraining. In order to provide such prompts, we further employ LLM-agnosticmodels to provide prompts that can describe image content and self-constructedquestion-answer pairs, which can effectively guide LLM to perform zero-shot VQAtasks. Img2Prompt offers the following benefits: 1) It can flexibly work withvarious LLMs to perform VQA. 2)~Without the needing of end-to-end training, itsignificantly reduces the cost of deploying LLM for zero-shot VQA tasks. 3) Itachieves comparable or better performance than methods relying on end-to-endtraining. For example, we outperform Flamingo \cite{Deepmind:Flamingo2022} by5.6\% on VQAv2. On the challenging A-OKVQA dataset, our method even outperformsfew-shot methods by as much as 20\%.

From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models