English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In long context scenarios, large language models (LLMs) face three mainchallenges: higher computational/financial cost, longer latency, and inferiorperformance. Some studies reveal that the performance of LLMs depends on boththe density and the position of the key information (question relevant) in theinput prompt. Inspired by these findings, we propose LongLLMLingua for promptcompression towards improving LLMs' perception of the key information tosimultaneously address the three challenges. We conduct evaluation on a widerange of long context scenarios including single-/multi-document QA, few-shotlearning, summarization, synthetic tasks, and code completion. The experimentalresults show that LongLLMLingua compressed prompt can derive higher performancewith much less cost. The latency of the end-to-end system is also reduced. Forexample, on NaturalQuestions benchmark, LongLLMLingua gains a performance boostof up to 17.1% over the original prompt with ~4x fewer tokens as input toGPT-3.5-Turbo. It can derive cost savings of \$28.5 and \$27.4 per 1,000samples from the LongBench and ZeroScrolls benchmark, respectively.Additionally, when compressing prompts of ~10k tokens at a compression rate of2x-10x, LongLLMLingua can speed up the end-to-end latency by 1.4x-3.8x. Ourcode is available at https://aka.ms/LLMLingua.

LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression