English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Models for Visual Question Answering (VQA) are notorious for their tendencyto rely on dataset biases, as the large and unbalanced diversity of questionsand concepts involved and tends to prevent models from learning to reason,leading them to perform educated guesses instead. In this paper, we claim thatthe standard evaluation metric, which consists in measuring the overallin-domain accuracy, is misleading. Since questions and concepts are unbalanced,this tends to favor models which exploit subtle training set statistics.Alternatively, naively introducing artificial distribution shifts between trainand test splits is also not completely satisfying. First, the shifts do notreflect real-world tendencies, resulting in unsuitable models; second, sincethe shifts are handcrafted, trained models are specifically designed for thisparticular setting, and do not generalize to other configurations. We proposethe GQA-OOD benchmark designed to overcome these concerns: we measure andcompare accuracy over both rare and frequent question-answer pairs, and arguethat the former is better suited to the evaluation of reasoning abilities,which we experimentally validate with models trained to more or less exploitbiases. In a large-scale study involving 7 VQA models and 3 bias reductiontechniques, we also experimentally demonstrate that these models fail toaddress questions involving infrequent concepts and provide recommendations forfuture directions of research.

Roses Are Red, Violets Are Blue... but Should Vqa Expect Them To?