English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Vision-language models (VLMs) classify the query video by calculating asimilarity score between the visual features and text-based class labelrepresentations. Recently, large language models (LLMs) have been used toenrich the text-based class labels by enhancing the descriptiveness of theclass names. However, these improvements are restricted to the text-basedclassifier only, and the query visual features are not considered. In thispaper, we propose a framework which combines pre-trained discriminative VLMswith pre-trained generative video-to-text and text-to-text models. We introducetwo key modifications to the standard zero-shot setting. First, we proposelanguage-guided visual feature enhancement and employ a video-to-text model toconvert the query video to its descriptive form. The resulting descriptionscontain vital visual cues of the query video, such as what objects are presentand their spatio-temporal interactions. These descriptive cues provideadditional semantic knowledge to VLMs to enhance their zeroshot performance.Second, we propose video-specific prompts to LLMs to generate more meaningfuldescriptions to enrich class label representations. Specifically, we introduceprompt techniques to create a Tree Hierarchy of Categories for class names,offering a higher-level action context for additional visual cues, Wedemonstrate the effectiveness of our approach in video understanding acrossthree different zero-shot settings: 1) video action recognition, 2)video-to-text and textto-video retrieval, and 3) time-sensitive video tasks.Consistent improvements across multiple benchmarks and with various VLMsdemonstrate the effectiveness of our proposed framework. Our code will be madepublicly available.

Videoprompter: an ensemble of foundational models for zero-shot video understanding