English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Traditional pruning methods are known to be challenging to work in LargeLanguage Models (LLMs) for Generative AI because of their unaffordable trainingprocess and large computational demands. For the first time, we introduce theinformation entropy of hidden state features into a pruning metric design,namely E-Sparse, to improve the accuracy of N:M sparsity on LLM. E-Sparseemploys the information richness to leverage the channel importance, andfurther incorporates several novel techniques to put it into effect: (1) itintroduces information entropy to enhance the significance of parameter weightsand input feature norms as a novel pruning metric, and performs N:M sparsitywithout modifying the remaining weights. (2) it designs global naive shuffleand local block shuffle to quickly optimize the information distribution andadequately cope with the impact of N:M sparsity on LLMs' accuracy. E-Sparse isimplemented as a Sparse-GEMM on FasterTransformer and runs on NVIDIA AmpereGPUs. Extensive experiments on the LLaMA family and OPT models show thatE-Sparse can significantly speed up the model inference over the dense model(up to 1.53X) and obtain significant memory saving (up to 43.52%), withacceptable accuracy loss.

E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsity