English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural retrievers based on pre-trained language models (PLMs), such asdual-encoders, have achieved promising performance on the task of open-domainquestion answering (QA). Their effectiveness can further reach newstate-of-the-arts by incorporating cross-architecture knowledge distillation.However, most of the existing studies just directly apply conventionaldistillation methods. They fail to consider the particular situation where theteacher and student have different structures. In this paper, we propose anovel distillation method that significantly advances cross-architecturedistillation for dual-encoders. Our method 1) introduces a self on-the-flydistillation method that can effectively distill late interaction (i.e.,ColBERT) to vanilla dual-encoder, and 2) incorporates a cascade distillationprocess to further improve the performance with a cross-encoder teacher.Extensive experiments are conducted to validate that our proposed solutionoutperforms strong baselines and establish a new state-of-the-art onopen-domain QA benchmarks.

ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrieval