English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transformer-based language models have taken many fields in NLP by storm.BERT and its derivatives dominate most of the existing evaluation benchmarks,including those for Word Sense Disambiguation (WSD), thanks to their ability incapturing context-sensitive semantic nuances. However, there is still littleknowledge about their capabilities and potential limitations in encoding andrecovering word senses. In this article, we provide an in-depth quantitativeand qualitative analysis of the celebrated BERT model with respect to lexicalambiguity. One of the main conclusions of our analysis is that BERT canaccurately capture high-level sense distinctions, even when a limited number ofexamples is available for each word sense. Our analysis also reveals that insome cases language models come close to solving coarse-grained noundisambiguation under ideal conditions in terms of availability of training dataand computing resources. However, this scenario rarely occurs in real-worldsettings and, hence, many practical challenges remain even in thecoarse-grained setting. We also perform an in-depth comparison of the two mainlanguage model based WSD strategies, i.e., fine-tuning and feature extraction,finding that the latter approach is more robust with respect to sense bias andit can better exploit limited available training data. In fact, the simplefeature extraction strategy of averaging contextualized embeddings provesrobust even using only three training sentences per word sense, with minimalimprovements obtained by increasing the size of this training data.

Analysis and Evaluation of Language Models for Word Sense Disambiguation