English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Dual encoders are now the dominant architecture for dense retrieval. Yet, wehave little understanding of how they represent text, and why this leads togood performance. In this work, we shed light on this question viadistributions over the vocabulary. We propose to interpret the vectorrepresentations produced by dual encoders by projecting them into the model'svocabulary space. We show that the resulting projections contain rich semanticinformation, and draw connection between them and sparse retrieval. We findthat this view can offer an explanation for some of the failure cases of denseretrievers. For example, we observe that the inability of models to handle tailentities is correlated with a tendency of the token distributions to forgetsome of the tokens of those entities. We leverage this insight and propose asimple way to enrich query and passage representations with lexical informationat inference time, and show that this significantly improves performancecompared to the original model in zero-shot settings, and specifically on theBEIR benchmark.

What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary