English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Since ChatGPT released its API for public use, the number of applicationsbuilt on top of commercial large language models (LLMs) increase exponentially.One popular usage of such models is leveraging its in-context learning abilityand generating responses given user queries leveraging knowledge obtained byretrieval augmentation. One problem of deploying commercial retrieval-augmentedLLMs is the cost due to the additionally retrieved context that largelyincreases the input token size of the LLMs. To mitigate this, we propose atoken compression scheme that includes two methods: summarization compressionand semantic compression. The first method applies a T5-based model that isfine-tuned by datasets generated using self-instruct containing samples withvarying lengths and reduce token size by doing summarization. The second methodfurther compresses the token size by removing words with lower impact on thesemantic. In order to adequately evaluate the effectiveness of the proposedmethods, we propose and utilize a dataset called Food-Recommendation DB (FRDB)focusing on food recommendation for women around pregnancy period or infants.Our summarization compression can reduce 65% of the retrieval token size withfurther 0.3% improvement on the accuracy; semantic compression provides a moreflexible way to trade-off the token size with performance, for which we canreduce the token size by 20% with only 1.6% of accuracy drop.

TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction