English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Converting a model's internals to text can yield human-understandableinsights about the model. Inspired by the recent success of training-freeapproaches for image captioning, we propose ZS-A2T, a zero-shot framework thattranslates the transformer attention of a given model into natural languagewithout requiring any training. We consider this in the context of VisualQuestion Answering (VQA). ZS-A2T builds on a pre-trained large language model(LLM), which receives a task prompt, question, and predicted answer, as inputs.The LLM is guided to select tokens which describe the regions in the inputimage that the VQA model attended to. Crucially, we determine this similarityby exploiting the text-image matching capabilities of the underlying VQA model.Our framework does not require any training and allows the drop-in replacementof different guiding sources (e.g. attribution instead of attention maps), orlanguage models. We evaluate this novel task on textual explanation datasetsfor VQA, giving state-of-the-art performances for the zero-shot setting onGQA-REX and VQA-X. Our code is available at:https://github.com/ExplainableML/ZS-A2T.

Zero-shot Translation of Attention Patterns in VQA Models to Natural Language