English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In text-video retrieval, the objective is to learn a cross-modal similarityfunction between a text and a video that ranks relevant text-video pairs higherthan irrelevant pairs. However, videos inherently express a much wider gamut ofinformation than texts. Instead, texts often capture sub-regions of entirevideos and are most semantically similar to certain frames within videos.Therefore, for a given text, a retrieval model should focus on the text's mostsemantically similar video sub-regions to make a more relevant comparison. Yet,most existing works aggregate entire videos without directly considering text.Common text-agnostic aggregations schemes include mean-pooling orself-attention over the frames, but these are likely to encode misleadingvisual information not described in the given text. To address this, we proposea cross-modal attention model called X-Pool that reasons between a text and theframes of a video. Our core mechanism is a scaled dot product attention for atext to attend to its most semantically similar frames. We then generate anaggregated video representation conditioned on the text's attention weightsover the frames. We evaluate our method on three benchmark datasets of MSR-VTT,MSVD and LSMDC, achieving new state-of-the-art results by up to 12% in relativeimprovement in Recall@1. Our findings thereby highlight the importance of jointtext-video reasoning to extract important visual cues according to text. Fullcode and demo can be found at: https://layer6ai-labs.github.io/xpool/

X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrieval