English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

High-dimensional token embeddings underpin Large Language Models (LLMs), asthey can capture subtle semantic information and significantly enhance themodelling of complex language patterns. However, the associated highdimensionality also introduces considerable model parameters, and aprohibitively high model storage. To address this issue, this work proposes anapproach based on the Tensor-Train Decomposition (TTD), where each tokenembedding is treated as a Matrix Product State (MPS) that can be efficientlycomputed in a distributed manner. The experimental results on GPT-2 demonstratethat, through our approach, the embedding layer can be compressed by a factorof up to 38.40 times, and when the compression factor is 3.31 times, evenproduced a better performance than the original GPT-2 model.

TensorGPT: Efficient Compression of the Embedding Layer in LLMs based on the Tensor-Train Decomposition