English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With the breakthrough of multi-modal large language models, answering complexvisual questions that demand advanced reasoning abilities and world knowledgehas become a much more important testbed for developing AI models than ever.However, equipping AI models with robust cross-modality reasoning abilityremains challenging since the cognition scheme of humans has not beenunderstood systematically. In this paper, we believe that if we can collectvisual clues in the given image as much as possible, we will recognize theimage more accurately, understand the question better, recall relevantknowledge more easily, and finally reason out the answer. We discover theserich visual clues by mining question-answer pairs in images and sending theminto multi-modal large language models as prompts. We call the proposed methodQ&A Prompts. Specifically, we first use the image-answer pairs and thecorresponding questions in the training set as inputs and outputs to train avisual question generation model. Then, we use an image tagging model toidentify various instances and send packaged image-tag pairs into the visualquestion generation model to generate relevant questions with the extractedimage tags as answers. Finally, we encode these generated question-answer pairsas prompts with a visual-aware prompting module and send them into pre-trainedmulti-modal large language models to reason out the final answers. Experimentalresults show that, compared with state-of-the-art methods, our Q&A Promptsachieves substantial improvements on the challenging visual question answeringdatasets requiring reasoning over diverse world knowledge, such as OK-VQA andA-OKVQA.

Q&A Prompts: Discovering Rich Visual Clues through Mining Question-Answer Prompts for VQA requiring Diverse World Knowledge