English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Retrieval-enhanced language models (LMs), which condition their predictionson text retrieved from large external datastores, have recently shownsignificant perplexity improvements compared to standard LMs. One suchapproach, the $k$NN-LM, interpolates any existing LM's predictions with theoutput of a $k$-nearest neighbors model and requires no additional training. Inthis paper, we explore the importance of lexical and semantic matching in thecontext of items retrieved by $k$NN-LM. We find two trends: (1) the presence oflarge overlapping $n$-grams between the datastore and evaluation set plays animportant factor in strong performance, even when the datastore is derived fromthe training data; and (2) the $k$NN-LM is most beneficial when retrieved itemshave high semantic similarity with the query. Based on our analysis, we definea new formulation of the $k$NN-LM that uses retrieval quality to assign theinterpolation coefficient. We empirically measure the effectiveness of ourapproach on two English language modeling datasets, Wikitext-103 and PG-19. Ourre-formulation of the $k$NN-LM is beneficial in both cases, and leads to nearly4% improvement in perplexity on the Wikitext-103 test set.

You can't pick your neighbors, or can you? When and how to rely on retrieval in the $k$NN-LM