English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

While transformers have shown great potential on video recognition with theirstrong capability of capturing long-range dependencies, they often suffer highcomputational costs induced by the self-attention to the huge number of 3Dtokens. In this paper, we present a new transformer architecture termedDualFormer, which can efficiently perform space-time attention for videorecognition. Concretely, DualFormer stratifies the full space-time attentioninto dual cascaded levels, i.e., to first learn fine-grained local interactionsamong nearby 3D tokens, and then to capture coarse-grained global dependenciesbetween the query token and global pyramid contexts. Different from existingmethods that apply space-time factorization or restrict attention computationswithin local windows for improving efficiency, our local-global stratificationstrategy can well capture both short- and long-range spatiotemporaldependencies, and meanwhile greatly reduces the number of keys and values inattention computation to boost efficiency. Experimental results verify thesuperiority of DualFormer on five video benchmarks against existing methods. Inparticular, DualFormer achieves 82.9%/85.2% top-1 accuracy on Kinetics-400/600with ~1000G inference FLOPs which is at least 3.2x fewer than existing methodswith similar performance. We have released the source code athttps://github.com/sail-sg/dualformer.

DualFormer: Local-Global Stratified Transformer for Efficient Video Recognition