English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

This paper studies masked autoencoder (MAE) video pre-training for varioustemporal matching-based downstream tasks, i.e., object-level tracking tasksincluding video object tracking (VOT) and video object segmentation (VOS),self-supervised visual correspondence learning, dense tracking tasks includingoptical flow estimation and long-term point tracking, and 3D point cloudtracking. Specifically, our work explores to provide a general representationto boost the temporal matching ability in various downstream tracking tasks. Toachieve this, we firstly find that a simple extension of MAE, which randomlymasks out frame patches in videos and reconstruct the frame pixels, heavilyrelies on spatial cues while ignoring temporal relations for framereconstruction, thus leading to sub-optimal temporal matching representations.To alleviate this, we propose DropMAE, which adaptively performsspatial-attention dropout in the frame reconstruction to facilitate temporalcorrespondence learning in videos. We obtain several important findings withDropMAE: 1) DropMAE is a strong and efficient temporal matching learner, whichachieves better fine-tuning results on matching-based tasks than theImageNet-based MAE with 2x faster pre-training speed. 2) DropMAE is effectivefor different tracking tasks, i.e., object-level matching tasks including VOTand VOS, dense tracking tasks including optical flow estimation and trackingany point (TAP), and even 3D tracking in the different modality of point clouddata. Since none exists, we build ViT-based trackers for different downstreamtracking tasks, and our pre-trained DropMAE model can be directly loaded inthese ViT-based trackers for fine-tuning without further modifications.Experiments on 6 downstream tracking tasks demonstrate the effectiveness ofDropMAE as a general pre-trained representation for diverse tracking tasks.

DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Tasks