English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In recent years, multi-modal transformers have shown significant progress inVision-Language tasks, such as Visual Question Answering (VQA), outperformingprevious architectures by a considerable margin. This improvement in VQA isoften attributed to the rich interactions between vision and language streams.In this work, we investigate the efficacy of co-attention transformer layers inhelping the network focus on relevant regions while answering the question. Wegenerate visual attention maps using the question-conditioned image attentionscores in these co-attention layers. We evaluate the effect of the followingcritical components on visual attention of a state-of-the-art VQA model: (i)number of object region proposals, (ii) question part of speech (POS) tags,(iii) question semantics, (iv) number of co-attention layers, and (v) answeraccuracy. We compare the neural network attention maps against human attentionmaps both qualitatively and quantitatively. Our findings indicate thatco-attention transformer modules are crucial in attending to relevant regionsof the image given a question. Importantly, we observe that the semanticmeaning of the question is not what drives visual attention, but specifickeywords in the question do. Our work sheds light on the function andinterpretation of co-attention transformer layers, highlights gaps in currentnetworks, and can guide the development of future VQA models and networks thatsimultaneously process visual and language streams.

On the Efficacy of Co-Attention Transformer Layers in Visual Question Answering