English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Most existing methods in vision-language retrieval match two modalities byeither comparing their global feature vectors which misses sufficientinformation and lacks interpretability, detecting objects in images or videosand aligning the text with fine-grained features which relies on complicatedmodel designs, or modeling fine-grained interaction via cross-attention uponvisual and textual tokens which suffers from inferior efficiency. To addressthese limitations, some recent works simply aggregate the token-wisesimilarities to achieve fine-grained alignment, but they lack intuitiveexplanations as well as neglect the relationships between token-level featuresand global representations with high-level semantics. In this work, we rethinkfine-grained cross-modal alignment and devise a new model-agnostic formulationfor it. We additionally demystify the recent popular works and subsume theminto our scheme. Furthermore, inspired by optimal transport theory, weintroduce TokenFlow, an instantiation of the proposed scheme. By modifying onlythe similarity function, the performance of our method is comparable to theSoTA algorithms with heavy model designs on major video-text retrievalbenchmarks. The visualization further indicates that TokenFlow successfullyleverages the fine-grained information and achieves better interpretability.

TokenFlow: Rethinking Fine-grained Cross-modal Alignment in Vision-Language Retrieval