English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large deep learning models have achieved impressive performance across arange of applications. However, their large memory requirements, includingparameter memory and activation memory, have become a significant challenge fortheir practical serving. While existing methods mainly address parametermemory, the importance of activation memory has been overlooked. Especially forlong input sequences, activation memory is expected to experience a significantexponential growth as the length of sequences increases. In this approach, wepropose AutoChunk, an automatic and adaptive compiler system that efficientlyreduces activation memory for long sequence inference by chunk strategies. Theproposed system generates chunk plans by optimizing through multiple stages. Ineach stage, the chunk search pass explores all possible chunk candidates andthe chunk selection pass identifies the optimal one. At runtime, AutoChunkemploys code generation to automatically apply chunk strategies. Theexperiments demonstrate that AutoChunk can reduce over 80\% of activationmemory while maintaining speed loss within 10%, extend max sequence length by3.2x to 11.7x, and outperform state-of-the-art methods by a large margin.

AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference