English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Most existing text-video retrieval methods focus on cross-modal matchingbetween the visual content of videos and textual query sentences. However, inreal-world scenarios, online videos are often accompanied by relevant textinformation such as titles, tags, and even subtitles, which can be utilized tomatch textual queries. This insight has motivated us to propose a novelapproach to text-video retrieval, where we directly generate associatedcaptions from videos using zero-shot video captioning with knowledge fromweb-scale pre-trained models (e.g., CLIP and GPT-2). Given the generatedcaptions, a natural question arises: what benefits do they bring to text-videoretrieval? To answer this, we introduce Cap4Video, a new framework thatleverages captions in three ways: i) Input data: video-caption pairs canaugment the training data. ii) Intermediate feature interaction: we performcross-modal feature interaction between the video and caption to produceenhanced video representations. iii) Output score: the Query-Caption matchingbranch can complement the original Query-Video matching branch for text-videoretrieval. We conduct comprehensive ablation studies to demonstrate theeffectiveness of our approach. Without any post-processing, Cap4Video achievesstate-of-the-art performance on four standard text-video retrieval benchmarks:MSR-VTT (51.4%), VATEX (66.6%), MSVD (51.8%), and DiDeMo (52.0%). The code isavailable at https://github.com/whwu95/Cap4Video .

Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?