English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transformer-based language models achieve high performance on various tasks,but we still lack understanding of the kind of linguistic knowledge they learnand rely on. We evaluate three models (BERT, RoBERTa, and ALBERT), testingtheir grammatical and semantic knowledge by sentence-level probing, diagnosticcases, and masked prediction tasks. We focus on relative clauses (in AmericanEnglish) as a complex phenomenon needing contextual information and antecedentidentification to be resolved. Based on a naturalistic dataset, probing showsthat all three models indeed capture linguistic knowledge about grammaticality,achieving high performance. Evaluation on diagnostic cases and maskedprediction tasks considering fine-grained linguistic knowledge, however, showspronounced model-specific weaknesses especially on semantic knowledge, stronglyimpacting models' performance. Our results highlight the importance of (a)modelcomparison in evaluation task and (b) building up claims of model performanceand the linguistic knowledge they capture beyond purely probing-basedevaluations.

A Closer Look at Linguistic Knowledge in Masked Language Models: The Case of Relative Clauses in American English