English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Neural networks, particularly Transformer-based architectures, have achievedsignificant performance improvements on several retrieval benchmarks. When theitems being retrieved are documents, the time and memory cost of employingTransformers over a full sequence of document terms can be prohibitive. Apopular strategy involves considering only the first n terms of the document.This can, however, result in a biased system that under retrieves longerdocuments. In this work, we propose a local self-attention which considers amoving window over the document terms and for each term attends only to otherterms in the same window. This local attention incurs a fraction of the computeand memory cost of attention over the whole document. The windowed approachalso leads to more compact packing of padded documents in minibatches resultingin additional savings. We also employ a learned saturation function and atwo-staged pooling strategy to identify relevant regions of the document. TheTransformer-Kernel pooling model with these changes can efficiently elicitrelevance information from documents with thousands of tokens. We benchmark ourproposed modifications on the document ranking task from the TREC 2019 DeepLearning track and observe significant improvements in retrieval quality aswell as increased retrieval of longer documents at moderate increase in computeand memory costs.

Local Self-Attention over Long Text for Efficient Document Retrieval