English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite exciting recent results showing vision-language systems' capacity toreason about images using natural language, their capacity for video reasoningremains under-explored. We motivate framing video reasoning as the sequentialunderstanding of a small number of keyframes, thereby leveraging the power androbustness of vision-language while alleviating the computational complexitiesof processing videos. To evaluate this novel application, we introduce VIP, aninference-time challenge dataset designed to explore models' reasoningcapabilities through video chain-of-thought. Inspired by visually descriptivescene plays, we propose two formats for keyframe description: unstructureddense captions and structured scene descriptions that identify the focus,action, mood, objects, and setting (FAMOuS) of the keyframe. To evaluate videoreasoning, we propose two tasks: Video Infilling and Video Prediction, whichtest abilities to generate multiple intermediate keyframes and predict futurekeyframes, respectively. We benchmark GPT-4, GPT-3, and VICUNA on VIP,demonstrate the performance gap in these complex video reasoning tasks, andencourage future work to prioritize language models for efficient andgeneralized video reasoning.

Let's Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought