English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Dense video captioning, a task of localizing meaningful moments andgenerating relevant captions for videos, often requires a large, expensivecorpus of annotated video segments paired with text. In an effort to minimizethe annotation cost, we propose ZeroTA, a novel method for dense videocaptioning in a zero-shot manner. Our method does not require any videos orannotations for training; instead, it localizes and describes events withineach input video at test time by optimizing solely on the input. This isaccomplished by introducing a soft moment mask that represents a temporalsegment in the video and jointly optimizing it with the prefix parameters of alanguage model. This joint optimization aligns a frozen language generationmodel (i.e., GPT-2) with a frozen vision-language contrastive model (i.e.,CLIP) by maximizing the matching score between the generated text and a momentwithin the video. We also introduce a pairwise temporal IoU loss to let a setof soft moment masks capture multiple distinct events within the video. Ourmethod effectively discovers diverse significant events within the video, withthe resulting captions appropriately describing these events. The empiricalresults demonstrate that ZeroTA surpasses zero-shot baselines and evenoutperforms the state-of-the-art few-shot method on the widely-used benchmarkActivityNet Captions. Moreover, our method shows greater robustness compared tosupervised methods when evaluated in out-of-domain scenarios. This researchprovides insight into the potential of aligning widely-used models, such aslanguage generation models and vision-language models, to unlock a newcapability: understanding temporal aspects of videos.

Zero-Shot Dense Video Captioning by Jointly Optimizing Text and Moment