English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vector quantization (VQ) based ANN indexes, such as Inverted File System(IVF) and Product Quantization (PQ), have been widely applied to embeddingbased document retrieval thanks to the competitive time and memory efficiency.Originally, VQ is learned to minimize the reconstruction loss, i.e., thedistortions between the original dense embeddings and the reconstructedembeddings after quantization. Unfortunately, such an objective is inconsistentwith the goal of selecting ground-truth documents for the input query, whichmay cause severe loss of retrieval quality. Recent works identify such adefect, and propose to minimize the retrieval loss through contrastivelearning. However, these methods intensively rely on queries with ground-truthdocuments, whose performance is limited by the insufficiency of labeled data.  In this paper, we propose Distill-VQ, which unifies the learning of IVF andPQ within a knowledge distillation framework. In Distill-VQ, the denseembeddings are leveraged as "teachers", which predict the query's relevance tothe sampled documents. The VQ modules are treated as the "students", which arelearned to reproduce the predicted relevance, such that the reconstructedembeddings may fully preserve the retrieval result of the dense embeddings. Bydoing so, Distill-VQ is able to derive substantial training signals from themassive unlabeled data, which significantly contributes to the retrievalquality. We perform comprehensive explorations for the optimal conduct ofknowledge distillation, which may provide useful insights for the learning ofVQ based ANN index. We also experimentally show that the labeled data is nolonger a necessity for high-quality vector quantization, which indicatesDistill-VQ's strong applicability in practice.

Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings