English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We introduce second-order vector representations of words, induced fromnearest neighborhood topological features in pre-trained contextual wordembeddings. We then analyze the effects of using second-order embeddings asinput features in two deep natural language processing models, for named entityrecognition and recognizing textual entailment, as well as a linear model forparaphrase recognition. Surprisingly, we find that nearest neighbor informationalone is sufficient to capture most of the performance benefits derived fromusing pre-trained word embeddings. Furthermore, second-order embeddings areable to handle highly heterogeneous data better than first-orderrepresentations, though at the cost of some specificity. Additionally,augmenting contextual embeddings with second-order information further improvesmodel performance in some cases. Due to variance in the random initializationsof word embeddings, utilizing nearest neighbor features from multiplefirst-order embedding samples can also contribute to downstream performancegains. Finally, we identify intriguing characteristics of second-orderembedding spaces for further research, including much higher density anddifferent semantic interpretations of cosine similarity.

Second-Order Word Embeddings from Nearest Neighbor Topological Features