English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The CoNLL-2003 English named entity recognition (NER) dataset has been widelyused to train and evaluate NER models for almost 20 years. However, it isunclear how well models that are trained on this 20-year-old data and developedover a period of decades using the same test set will perform when applied onmodern data. In this paper, we evaluate the generalization of over 20 differentmodels trained on CoNLL-2003, and show that NER models have very differentgeneralization. Surprisingly, we find no evidence of performance degradation inpre-trained Transformers, such as RoBERTa and T5, even when fine-tuned usingdecades-old data. We investigate why some models generalize well to new datawhile others do not, and attempt to disentangle the effects of temporal driftand overfitting due to test reuse. Our analysis suggests that mostdeterioration is due to temporal mismatch between the pre-training corpora andthe downstream test sets. We found that four factors are important for goodgeneralization: model architecture, number of parameters, time period of thepre-training corpus, in addition to the amount of fine-tuning data. We suggestcurrent evaluation methods have, in some sense, underestimated progress on NERover the past 20 years, as NER models have not only improved on the originalCoNLL-2003 test set, but improved even more on modern data. Our datasets can befound at https://github.com/ShuhengL/acl2023_conllpp.

Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023?