English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Unsupervised sentence embedding aims to obtain the most appropriate embeddingfor a sentence to reflect its semantic. Contrastive learning has beenattracting developing attention. For a sentence, current models utilize diversedata augmentation methods to generate positive samples, while consider otherindependent sentences as negative samples. Then they adopt InfoNCE loss to pullthe embeddings of positive pairs gathered, and push those of negative pairsscattered. Although these models have made great progress on sentenceembedding, we argue that they may suffer from feature suppression. The modelsfail to distinguish and decouple textual similarity and semantic similarity.And they may overestimate the semantic similarity of any pairs with similartextual regardless of the actual semantic difference between them. This isbecause positive pairs in unsupervised contrastive learning come with similarand even the same textual through data augmentation. To alleviate featuresuppression, we propose contrastive learning for unsupervised sentenceembedding with soft negative samples (SNCSE). Soft negative samples sharehighly similar textual but have surely and apparently different semantic withthe original samples. Specifically, we take the negation of original sentencesas soft negative samples, and propose Bidirectional Margin Loss (BML) tointroduce them into traditional contrastive learning framework, which merelyinvolves positive and negative samples. Our experimental results show thatSNCSE can obtain state-of-the-art performance on semantic textual similarity(STS) task with average Spearman's correlation coefficient of 78.97% onBERTbase and 79.23% on RoBERTabase. Besides, we adopt rank-based error analysismethod to detect the weakness of SNCSE for future study.

SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samples