English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we propose a novel multi-modal framework for Scene Text VisualQuestion Answering (STVQA), which requires models to read scene text in imagesfor question answering. Apart from text or visual objects, which could existindependently, scene text naturally links text and visual modalities togetherby conveying linguistic semantics while being a visual object in an imagesimultaneously. Different to conventional STVQA models which take thelinguistic semantics and visual semantics in scene text as two separatefeatures, in this paper, we propose a paradigm of "Locate Then Generate" (LTG),which explicitly unifies this two semantics with the spatial bounding box as abridge connecting them. Specifically, at first, LTG locates the region in animage that may contain the answer words with an answer location module (ALM)consisting of a region proposal network and a language refinement network, bothof which can transform to each other with one-to-one mapping via the scene textbounding box. Next, given the answer words selected by ALM, LTG generates areadable answer sequence with an answer generation module (AGM) based on apre-trained language model. As a benefit of the explicit alignment of thevisual and linguistic semantics, even without any scene text based pre-trainingtasks, LTG can boost the absolute accuracy by +6.06% and +6.92% on the TextVQAdataset and the ST-VQA dataset respectively, compared with a non-pre-trainingbaseline. We further demonstrate that LTG effectively unifies visual and textmodalities through the spatial bounding box connection, which isunderappreciated in previous methods.

Locate Then Generate: Bridging Vision and Language with Bounding Box for Scene-Text VQA