English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper presents SimVTP: a Simple Video-Text Pretraining framework viamasked autoencoders. We randomly mask out the spatial-temporal tubes of inputvideo and the word tokens of input text and then feed them into a unifiedautencoder to reconstruct the missing pixels and words. Our SimVTP has severalproperties: 1) Thanks to the unified autoencoder, SimVTP reconstructs themasked signal of one modality with the help from another modality, whichimplicitly learns the cross-modal alignment between video tubes and texttokens. 2) SimVTP not only benefits from a high video masking ratio (e.g. 90%)due to the temporal redundancy of video, but also needs a high text maskingratio (e.g. 75%), which is much higher than BERT (e.g. 15%), to achieve optimalperformance. This is because the aid of video modality makes textreconstruction less challenging, which thus needs a higher mask ratio to makethe pretext harder for useful feature learning. 3) Equipping SimVTP withvideo-text contrastive learning (VTC) and video-text matching (VTM), which aretwo commonly used cross-modal training strategies, could further improve thetransferable performance significantly. 4) SimVTP is dataefficent, e.g.,pre-training only on 10% data of WebVid-2M, SimVTP achieves surprisingly goodresults (43.8 R@1) on MSRVTT, which is far above recent state-of-the-artmethods pre-trained on both CC3M and WebVid-2M. We transfer our pre-trainedmodel to various downstream tasks and achieve superior performance. The codesand models will be released at https://github.com/mayuelala/SimVTP.

SimVTP: Simple Video Text Pre-training with Masked Autoencoders