English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vision Transformers (ViTs) have revolutionized the field of computer vision,yet their deployments on resource-constrained devices remain challenging due tohigh computational demands. To expedite pre-trained ViTs, token pruning andtoken merging approaches have been developed, which aim at reducing the numberof tokens involved in the computation. However, these methods still have somelimitations, such as image information loss from pruned tokens and inefficiencyin the token-matching process. In this paper, we introduce a novel Graph-basedToken Propagation (GTP) method to resolve the challenge of balancing modelefficiency and information preservation for efficient ViTs. Inspired by graphsummarization algorithms, GTP meticulously propagates less significant tokens'information to spatially and semantically connected tokens that are of greaterimportance. Consequently, the remaining few tokens serve as a summarization ofthe entire token graph, allowing the method to reduce computational complexitywhile preserving essential information of eliminated tokens. Combined with aninnovative token selection strategy, GTP can efficiently identify image tokensto be propagated. Extensive experiments have validated GTP's effectiveness,demonstrating both efficiency and performance improvements. Specifically, GTPdecreases the computational complexity of both DeiT-S and DeiT-B by up to 26%with only a minimal 0.3% accuracy drop on ImageNet-1K without finetuning, andremarkably surpasses the state-of-the-art token merging method on variousbackbones at an even faster inference speed. The source code is available athttps://github.com/Ackesnal/GTP-ViT.

GTP-ViT: Efficient Vision Transformers via Graph-based Token Propagation