English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Contextualized word embeddings in language models have given much advance toNLP. Intuitively, sentential information is integrated into the representationof words, which can help model polysemy. However, context sensitivity alsoleads to the variance of representations, which may break the semanticconsistency for synonyms. We quantify how much the contextualized embeddings ofeach word sense vary across contexts in typical pre-trained models. Resultsshow that contextualized embeddings can be highly consistent across contexts.In addition, part-of-speech, number of word senses, and sentence length have aninfluence on the variance of sense representations. Interestingly, we find thatword representations are position-biased, where the first words in differentcontexts tend to be more similar. We analyze such a phenomenon and also proposea simple way to alleviate such bias in distance-based word sense disambiguationsettings.

Lost in Context? On the Sense-wise Variance of Contextualized Word Embeddings