English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-trained Language Model (PLM) is nowadays the mainstay of UnsupervisedSentence Representation Learning (USRL). However, PLMs are sensitive to thefrequency information of words from their pre-training corpora, resulting inanisotropic embedding space, where the embeddings of high-frequency words areclustered but those of low-frequency words disperse sparsely. This anisotropicphenomenon results in two problems of similarity bias and information bias,lowering the quality of sentence embeddings. To solve the problems, wefine-tune PLMs by leveraging the frequency information of words and propose anovel USRL framework, namely Sentence Representation Learning withFrequency-induced Adversarial tuning and Incomplete sentence filtering(SLT-FAI). We calculate the word frequencies over the pre-training corpora ofPLMs and assign words thresholding frequency labels. With them, (1) weincorporate a similarity discriminator used to distinguish the embeddings ofhigh-frequency and low-frequency words, and adversarially tune the PLM with it,enabling to achieve uniformly frequency-invariant embedding space; and (2) wepropose a novel incomplete sentence detection task, where we incorporate aninformation discriminator to distinguish the embeddings of original sentencesand incomplete sentences by randomly masking several low-frequency words,enabling to emphasize the more informative low-frequency words. Our SLT-FAI isa flexible and plug-and-play framework, and it can be integrated with existingUSRL techniques. We evaluate SLT-FAI with various backbones on benchmarkdatasets. Empirical results indicate that SLT-FAI can be superior to theexisting USRL baselines. Our code is released in\url{https://github.com/wangbing1416/SLT-FAI}.

Unsupervised Sentence Representation Learning with Frequency-induced Adversarial Tuning and Incomplete Sentence Filtering