English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose a novel multimodal architecture for Scene Text Visual QuestionAnswering (STVQA), named Layout-Aware Transformer (LaTr). The task of STVQArequires models to reason over different modalities. Thus, we first investigatethe impact of each modality, and reveal the importance of the language module,especially when enriched with layout information. Accounting for this, wepropose a single objective pre-training scheme that requires only text andspatial cues. We show that applying this pre-training scheme on scanneddocuments has certain advantages over using natural images, despite the domaingap. Scanned documents are easy to procure, text-dense and have a variety oflayouts, helping the model learn various spatial cues (e.g. left-of, belowetc.) by tying together language and layout information. Compared to existingapproaches, our method performs vocabulary-free decoding and, as shown,generalizes well beyond the training vocabulary. We further demonstrate thatLaTr improves robustness towards OCR errors, a common reason for failure casesin STVQA. In addition, by leveraging a vision transformer, we eliminate theneed for an external object detector. LaTr outperforms state-of-the-art STVQAmethods on multiple datasets. In particular, +7.6% on TextVQA, +10.8% on ST-VQAand +4.0% on OCR-VQA (all absolute accuracy numbers).

LaTr: Layout-Aware Transformer for Scene-Text VQA