English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transformers have become the primary backbone of the computer visioncommunity due to their impressive performance. However, the unfriendlycomputation cost impedes their potential in the video recognition domain. Tooptimize the speed-accuracy trade-off, we propose Semantic-aware TemporalAccumulation score (STA) to prune spatio-temporal tokens integrally. STA scoreconsiders two critical factors: temporal redundancy and semantic importance.The former depicts a specific region based on whether it is a new occurrence ora seen entity by aggregating token-to-token similarity in consecutive frameswhile the latter evaluates each token based on its contribution to the overallprediction. As a result, tokens with higher scores of STA carry more temporalredundancy as well as lower semantics thus being pruned. Based on the STAscore, we are able to progressively prune the tokens without introducing anyadditional parameters or requiring further re-training. We directly apply theSTA module to off-the-shelf ViT and VideoSwin backbones, and the empiricalresults on Kinetics-400 and Something-Something V2 achieve over 30% computationreduction with a negligible ~0.2% accuracy drop. The code is released athttps://github.com/Mark12Ding/STA.

Prune Spatio-temporal Tokens by Semantic-aware Temporal Accumulation