English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deployment of Transformer models on edge devices is becoming increasinglychallenging due to the exponentially growing inference cost that scalesquadratically with the number of tokens in the input sequence. Token pruning isan emerging solution to address this challenge due to its ease of deployment onvarious Transformer backbones. However, most token pruning methods requirecomputationally expensive fine-tuning, which is undesirable in many edgedeployment cases. In this work, we propose Zero-TPrune, the first zero-shotmethod that considers both the importance and similarity of tokens inperforming token pruning. It leverages the attention graph of pre-trainedTransformer models to produce an importance distribution for tokens via ourproposed Weighted Page Rank (WPR) algorithm. This distribution further guidestoken partitioning for efficient similarity-based pruning. Due to theelimination of the fine-tuning overhead, Zero-TPrune can prune large models atnegligible computational cost, switch between different pruning configurationsat no computational cost, and perform hyperparameter tuning efficiently. Weevaluate the performance of Zero-TPrune on vision tasks by applying it tovarious vision Transformer backbones and testing them on ImageNet. Without anyfine-tuning, Zero-TPrune reduces the FLOPs cost of DeiT-S by 34.7\% andimproves its throughput by 45.3\% with only 0.4\% accuracy loss. Compared withstate-of-the-art pruning methods that require fine-tuning, Zero-TPrune not onlyeliminates the need for fine-tuning after pruning but also does so with only0.1\% accuracy loss. Compared with state-of-the-art fine-tuning-free pruningmethods, Zero-TPrune reduces accuracy loss by up to 49\% with the same orhigher throughput.

Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformers