English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large-scale image-text contrastive pre-training models, such as CLIP, havebeen demonstrated to effectively learn high-quality multimodal representations.However, there is limited research on learning video-text representations forgeneral video multimodal tasks based on these powerful features. Towards thisgoal, we propose a novel video-text pre-training method dubbed VLAB: VideoLanguage pre-training by feature Adapting and Blending, which transfers CLIPrepresentations to video pre-training tasks and develops unified videomultimodal models for a wide range of video-text tasks. Specifically, VLAB isfounded on two key strategies: feature adapting and feature blending. In theformer, we introduce a new video adapter module to address CLIP's deficiency inmodeling temporal information and extend the model's capability to encompassboth contrastive and generative tasks. In the latter, we propose an end-to-endtraining method that further enhances the model's performance by exploiting thecomplementarity of image and video features. We validate the effectiveness andversatility of VLAB through extensive experiments on highly competitive videomultimodal tasks, including video text retrieval, video captioning, and videoquestion answering. Remarkably, VLAB outperforms competing methodssignificantly and sets new records in video question answering on MSRVTT, MSVD,and TGIF datasets. It achieves an accuracy of 49.6, 61.0, and 79.0,respectively. Codes and models will be released.

VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending