English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Video-language pre-training is crucial for learning powerful multi-modalrepresentation. However, it typically requires a massive amount of computation.In this paper, we develop SMAUG, an efficient pre-training framework forvideo-language models. The foundation component in SMAUG is maskedautoencoders. Different from prior works which only mask textual inputs, ourmasking strategy considers both visual and textual modalities, providing abetter cross-modal alignment and saving more pre-training costs. On top ofthat, we introduce a space-time token sparsification module, which leveragescontext information to further select only "important" spatial regions andtemporal frames for pre-training. Coupling all these designs allows our methodto enjoy both competitive performances on text-to-video retrieval and videoquestion answering tasks, and much less pre-training costs by 1.9X or more. Forexample, our SMAUG only needs about 50 NVIDIA A6000 GPU hours for pre-trainingto attain competitive performances on these two video-language tasks across sixpopular benchmarks.

SMAUG: Sparse Masked Autoencoder for Efficient Video-Language Pre-training