English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Knowledge-based Visual Question Answering (VQA) expects models to rely onexternal knowledge for robust answer prediction. Though significant it is, thispaper discovers several leading factors impeding the advancement of currentstate-of-the-art methods. On the one hand, methods which exploit the explicitknowledge take the knowledge as a complement for the coarsely trained VQAmodel. Despite their effectiveness, these approaches often suffer from noiseincorporation and error propagation. On the other hand, pertaining to theimplicit knowledge, the multi-modal implicit knowledge for knowledge-based VQAstill remains largely unexplored. This work presents a unified end-to-endretriever-reader framework towards knowledge-based VQA. In particular, we shedlight on the multi-modal implicit knowledge from vision-language pre-trainingmodels to mine its potential in knowledge reasoning. As for the noise problemencountered by the retrieval operation on explicit knowledge, we design a novelscheme to create pseudo labels for effective knowledge supervision. This schemeis able to not only provide guidance for knowledge retrieval, but also dropthese instances potentially error-prone towards question answering. To validatethe effectiveness of the proposed method, we conduct extensive experiments onthe benchmark dataset. The experimental results reveal that our methodoutperforms existing baselines by a noticeable margin. Beyond the reportednumbers, this paper further spawns several insights on knowledge utilizationfor future research with some empirical findings.

A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA