English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

How to learn highly compact yet effective sentence representation?Pre-trained language models have been effective in many NLP tasks. However,these models are often huge and produce large sentence embeddings. Moreover,there is a big performance gap between large and small models. In this paper,we propose Homomorphic Projective Distillation (HPD) to learn compressedsentence embeddings. Our method augments a small Transformer encoder model withlearnable projection layers to produce compact representations while mimickinga large pre-trained language model to retain the sentence representationquality. We evaluate our method with different model sizes on both semantictextual similarity (STS) and semantic retrieval (SR) tasks. Experiments showthat our method achieves 2.7-4.5 points performance gain on STS tasks comparedwith previous best representations of the same size. In SR tasks, our methodimproves retrieval speed (8.2$\times$) and memory usage (8.0$\times$) comparedwith state-of-the-art large models.

Compressing Sentence Representation for Semantic Retrieval via Homomorphic Projective Distillation