English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent studies have proposed using large language models (LLMs) for sentenceembeddings. However, most existing LLMs are built with an autoregressivearchitecture that primarily captures forward dependencies while neglectingbackward dependencies. Previous work has highlighted the importance of backwarddependencies in improving sentence embeddings. To address this issue, in thispaper, we first present quantitative evidence demonstrating the limitedlearning of backward dependencies in LLMs. Then, we propose a novel approachcalled Dependency-Enhanced Large Language Model (DeeLM) to improve sentenceembeddings. Specifically, we found a turning point in LLMs, where surpassingspecific LLM layers leads to a significant performance drop in the semantictextual similarity (STS) task. STS is a crucial task for evaluating sentenceembeddings. We then extract the layers after the turning point to make thembidirectional, allowing for the learning of backward dependencies. Extensiveexperiments demonstrate that DeeLM outperforms baselines and achievesstate-of-the-art performance across various STS tasks.

DeeLM: Dependency-enhanced Large Language Model for Sentence Embeddings