English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Visual question answering (VQA) requires systems to perform concept-levelreasoning by unifying unstructured (e.g., the context in question and answer;"QA context") and structured (e.g., knowledge graph for the QA context andscene; "concept graph") multimodal knowledge. Existing works typically combinea scene graph and a concept graph of the scene by connecting correspondingvisual nodes and concept nodes, then incorporate the QA context representationto perform question answering. However, these methods only perform aunidirectional fusion from unstructured knowledge to structured knowledge,limiting their potential to capture joint reasoning over the heterogeneousmodalities of knowledge. To perform more expressive reasoning, we proposeVQA-GNN, a new VQA model that performs bidirectional fusion betweenunstructured and structured multimodal knowledge to obtain unified knowledgerepresentations. Specifically, we inter-connect the scene graph and the conceptgraph through a super node that represents the QA context, and introduce a newmultimodal GNN technique to perform inter-modal message passing for reasoningthat mitigates representational gaps between modalities. On two challenging VQAtasks (VCR and GQA), our method outperforms strong baseline VQA methods by 3.2%on VCR (Q-AR) and 4.6% on GQA, suggesting its strength in performingconcept-level reasoning. Ablation studies further demonstrate the efficacy ofthe bidirectional fusion and multimodal GNN method in unifying unstructured andstructured multimodal knowledge.

VQA-GNN: Reasoning with Multimodal Knowledge via Graph Neural Networks for Visual Question Answering