English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Contrastive learning has been studied for improving the performance oflearning sentence embeddings. The current state-of-the-art method is theSimCSE, which takes dropout as the data augmentation method and feeds apre-trained transformer encoder the same input sentence twice. Thecorresponding outputs, two sentence embeddings derived from the same sentencewith different dropout masks, can be used to build a positive pair. A networkbeing applied with a dropout mask can be regarded as a sub-network of itsef,whose expected scale is determined by the dropout rate. In this paper, we pushsub-networks with different expected scales learn similar embedding for thesame sentence. SimCSE failed to do so because they fixed the dropout rate to atuned hyperparameter. We achieve this by sampling dropout rate from adistribution eatch forward process. As this method may make optimizationharder, we also propose a simple sentence-wise mask strategy to sample moresub-networks. We evaluated the proposed S-SimCSE on several popular semantictext similarity datasets. Experimental results show that S-SimCSE outperformsthe state-of-the-art SimCSE more than $1\%$ on BERT$_{base}$

S-SimCSE: Sampled Sub-networks for Contrastive Learning of Sentence Embedding