English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

What makes good video representations for video understanding, such asanticipating future activities, or answering video-conditioned questions? Whileearlier approaches focus on end-to-end learning directly from video pixels, wepropose to revisit text-based representations, such as discrete action labels,or free-form video captions, which are interpretable and can be directlyconsumed by large language models (LLMs). Intuitively, different videounderstanding tasks may require representations that are complementary and atdifferent granularities. To this end, we propose versatile action models(Vamos), a learning framework powered by a large language model as the"reasoner", and can flexibly leverage visual embeddings, action labels, andfree-form descriptions extracted from videos as its input. We evaluate Vamos onfour complementary video understanding benchmarks, Ego4D, Next-QA, IntentQA,and EgoSchema, on its capability to model temporal dynamics, encode visualhistory, and perform reasoning. Surprisingly, we observe that text-basedrepresentations consistently achieve competitive performance on all benchmarks,and that visual embeddings provide marginal or no performance improvement,demonstrating the effectiveness of text-based video representation in the LLMera. We perform extensive ablation study and qualitative analysis to supportour observations, and achieve state-of-the-art performance on three benchmarks.

Vamos: Versatile Action Models for Video Understanding