English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Question answering (QA) models have shown compelling results in the task ofMachine Reading Comprehension (MRC). Recently these systems have proved toperform better than humans on held-out test sets of datasets e.g. SQuAD, buttheir robustness is not guaranteed. The QA model's brittleness is exposed whenevaluated on adversarial generated examples by a performance drop. In thisstudy, we explore the robustness of MRC models to entity renaming, withentities from low-resource regions such as Africa. We propose EntSwap, a methodfor test-time perturbations, to create a test set whose entities have beenrenamed. In particular, we rename entities of type: country, person,nationality, location, organization, and city, to create AfriSQuAD2. Using theperturbed test set, we evaluate the robustness of three popular MRC models. Wefind that compared to base models, large models perform well comparatively onnovel entities. Furthermore, our analysis indicates that entity type personhighly challenges the MRC models' performance.

Evaluating the Robustness of Machine Reading Comprehension Models to Low Resource Entity Renaming