English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vision Transformers (ViTs) have emerged as state-of-the-art models forvarious vision tasks recently. However, their heavy computation costs remaindaunting for resource-limited devices. Consequently, researchers have dedicatedthemselves to compressing redundant information in ViTs for acceleration.However, they generally sparsely drop redundant image tokens by token pruningor brutally remove channels by channel pruning, leading to a sub-optimalbalance between model performance and inference speed. They are alsodisadvantageous in transferring compressed models to downstream vision tasksthat require the spatial structure of images, such as semantic segmentation. Totackle these issues, we propose a joint compression method for ViTs that offersboth high accuracy and fast inference speed, while also maintaining favorabletransferability to downstream tasks (CAIT). Specifically, we introduce anasymmetric token merging (ATME) strategy to effectively integrate neighboringtokens. It can successfully compress redundant token information whilepreserving the spatial structure of images. We further employ a consistentdynamic channel pruning (CDCP) strategy to dynamically prune unimportantchannels in ViTs. Thanks to CDCP, insignificant channels in multi-headself-attention modules of ViTs can be pruned uniformly, greatly enhancing themodel compression. Extensive experiments on benchmark datasets demonstrate thatour proposed method can achieve state-of-the-art performance across variousViTs. For example, our pruned DeiT-Tiny and DeiT-Small achieve speedups of1.7$\times$ and 1.9$\times$, respectively, without accuracy drops on ImageNet.On the ADE20k segmentation dataset, our method can enjoy up to 1.31$\times$speedups with comparable mIoU. Our code will be publicly available.

CAIT: Triple-Win Compression towards High Accuracy, Fast Inference, and Favorable Transferability For ViTs