English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Given the enormous number of instructional videos available online, learninga diverse array of multi-step task models from videos is an appealing goal. Weintroduce a new pre-trained video model, VideoTaskformer, focused onrepresenting the semantics and structure of instructional videos. We pre-trainVideoTaskformer using a simple and effective objective: predicting weaklysupervised textual labels for steps that are randomly masked out from aninstructional video (masked step modeling). Compared to prior work which learnsstep representations locally, our approach involves learning them globally,leveraging video of the entire surrounding task as context. From these learnedrepresentations, we can verify if an unseen video correctly executes a giventask, as well as forecast which steps are likely to be taken after a givenstep. We introduce two new benchmarks for detecting mistakes in instructionalvideos, to verify if there is an anomalous step and if steps are executed inthe right order. We also introduce a long-term forecasting benchmark, where thegoal is to predict long-range future steps from a given step. Our methodoutperforms previous baselines on these tasks, and we believe the tasks will bea valuable way for the community to measure the quality of steprepresentations. Additionally, we evaluate VideoTaskformer on 3 existingbenchmarks -- procedural activity recognition, step classification, and stepforecasting -- and demonstrate on each that our method outperforms existingbaselines and achieves new state-of-the-art performance.

Learning and Verification of Task Structure in Instructional Videos