English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We consider the large-scale query-document retrieval problem: given a query(e.g., a question), return the set of relevant documents (e.g., paragraphscontaining the answer) from a large document corpus. This problem is oftensolved in two steps. The retrieval phase first reduces the solution space,returning a subset of candidate documents. The scoring phase then re-ranks thedocuments. Critically, the retrieval algorithm not only desires high recall butalso requires to be highly efficient, returning candidates in time sublinear tothe number of documents. Unlike the scoring phase witnessing significantadvances recently due to the BERT-style pre-training tasks on cross-attentionmodels, the retrieval phase remains less well studied. Most previous works relyon classic Information Retrieval (IR) methods such as BM-25 (token matching +TF-IDF weights). These models only accept sparse handcrafted features and cannot be optimized for different downstream tasks of interest. In this paper, weconduct a comprehensive study on the embedding-based retrieval models. We showthat the key ingredient of learning a strong embedding-based Transformer modelis the set of pre-training tasks. With adequately designed paragraph-levelpre-training tasks, the Transformer models can remarkably improve over thewidely-used BM-25 as well as embedding models without Transformers. Theparagraph-level pre-training tasks we studied are Inverse Cloze Task (ICT),Body First Selection (BFS), Wiki Link Prediction (WLP), and the combination ofall three.

Pre-training Tasks for Embedding-based Large-scale Retrieval