English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Contrastive learning has been attracting much attention for learningunsupervised sentence embeddings. The current state-of-the-art unsupervisedmethod is the unsupervised SimCSE (unsup-SimCSE). Unsup-SimCSE takes dropout asa minimal data augmentation method, and passes the same input sentence to apre-trained Transformer encoder (with dropout turned on) twice to obtain thetwo corresponding embeddings to build a positive pair. As the lengthinformation of a sentence will generally be encoded into the sentenceembeddings due to the usage of position embedding in Transformer, each positivepair in unsup-SimCSE actually contains the same length information. And thusunsup-SimCSE trained with these positive pairs is probably biased, which wouldtend to consider that sentences of the same or similar length are more similarin semantics. Through statistical observations, we find that unsup-SimCSE doeshave such a problem. To alleviate it, we apply a simple repetition operation tomodify the input sentence, and then pass the input sentence and its modifiedcounterpart to the pre-trained Transformer encoder, respectively, to get thepositive pair. Additionally, we draw inspiration from the community of computervision and introduce a momentum contrast, enlarging the number of negativepairs without additional calculations. The proposed two modifications areapplied on positive and negative pairs separately, and build a new sentenceembedding method, termed Enhanced Unsup-SimCSE (ESimCSE). We evaluate theproposed ESimCSE on several benchmark datasets w.r.t the semantic textsimilarity (STS) task. Experimental results show that ESimCSE outperforms thestate-of-the-art unsup-SimCSE by an average Spearman correlation of 2.02% onBERT-base.

ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding