English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While the success of pre-trained language models has largely eliminated theneed for high-quality static word vectors in many NLP applications, suchvectors continue to play an important role in tasks where words need to bemodelled in the absence of linguistic context. In this paper, we explore howthe contextualised embeddings predicted by BERT can be used to producehigh-quality word vectors for such domains, in particular related to knowledgebase completion, where our focus is on capturing the semantic properties ofnouns. We find that a simple strategy of averaging the contextualisedembeddings of masked word mentions leads to vectors that outperform the staticword vectors learned by BERT, as well as those from standard word embeddingmodels, in property induction tasks. We notice in particular that maskingtarget words is critical to achieve this strong performance, as the resultingvectors focus less on idiosyncratic properties and more on general semanticproperties. Inspired by this view, we propose a filtering strategy which isaimed at removing the most idiosyncratic mention vectors, allowing us to obtainfurther performance gains in property induction.

Modelling General Properties of Nouns by Selectively Averaging Contextualised Embeddings