English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Advances in the Visually-rich Document Understanding (VrDU) field andparticularly the Key-Information Extraction (KIE) task are marked with theemergence of efficient Transformer-based approaches such as the LayoutLMmodels. Despite the good performance of KIE models when fine-tuned on publicbenchmarks, they still struggle to generalize on complex real-life use-caseslacking sufficient document annotations. Our research highlighted that KIEstandard benchmarks such as SROIE and FUNSD contain significant similaritybetween training and testing documents and can be adjusted to better evaluatethe generalization of models. In this work, we designed experiments to quantifythe information redundancy in public benchmarks, revealing a 75% templatereplication in SROIE official test set and 16% in FUNSD. We also proposedresampling strategies to provide benchmarks more representative of thegeneralization ability of models. We showed that models not suited for documentanalysis struggle on the adjusted splits dropping on average 10,5% F1 score onSROIE and 3.5% on FUNSD compared to multi-modal models dropping only 7,5% F1 onSROIE and 0.5% F1 on FUNSD.

Information Redundancy and Biases in Public Document Information Extraction Benchmarks