English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper introduces a novel approach to enhance the capabilities of LargeLanguage Models (LLMs) in processing and understanding extensive textsequences, a critical aspect in applications requiring deep comprehension andsynthesis of large volumes of information. Recognizing the inherent challengesin extending the context window for LLMs, primarily built on Transformerarchitecture, we propose a new model architecture, referred to as Zebra. Thisarchitecture efficiently manages the quadratic time and memory complexityissues associated with full attention in the Transformer by employing groupedlocal-global attention layers. Our model, akin to a zebra's alternatingstripes, balances local and global attention layers, significantly reducingcomputational requirements and memory consumption. Comprehensive experiments,including pretraining from scratch, continuation of long context adaptationtraining, and long instruction tuning, are conducted to evaluate the Zebra'sperformance. The results show that Zebra achieves comparable or superiorperformance on both short and long sequence benchmarks, while also enhancingtraining and inference efficiency.

Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention