English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Machine Reading Comprehension (MRC) is an important testbed for evaluatingmodels' natural language understanding (NLU) ability. There has been rapidprogress in this area, with new models achieving impressive performance onvarious benchmarks. However, existing benchmarks only evaluate models onin-domain test sets without considering their robustness under test-timeperturbations or adversarial attacks. To fill this important gap, we constructAdvRACE (Adversarial RACE), a new model-agnostic benchmark for evaluating therobustness of MRC models under four different types of adversarial attacks,including our novel distractor extraction and generation attacks. We show thatstate-of-the-art (SOTA) models are vulnerable to all of these attacks. Weconclude that there is substantial room for building more robust MRC models andour benchmark can help motivate and measure progress in this area. We releaseour data and code at https://github.com/NoviScl/AdvRACE .

Benchmarking Robustness of Machine Reading Comprehension Models