English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In this paper we present an approach for localizing steps of proceduralactivities in narrated how-to videos. To deal with the scarcity of labeled dataat scale, we source the step descriptions from a language knowledge base(wikiHow) containing instructional articles for a large variety of proceduraltasks. Without any form of manual supervision, our model learns to temporallyground the steps of procedural articles in how-to videos by matching threemodalities: frames, narrations, and step descriptions. Specifically, our methodaligns steps to video by fusing information from two distinct pathways: i) {\emdirect} alignment of step descriptions to frames, ii) {\em indirect} alignmentobtained by composing steps-to-narrations with narrations-to-videocorrespondences. Notably, our approach performs global temporal grounding ofall steps in an article at once by exploiting order information, and is trainedwith step pseudo-labels which are iteratively refined and aggressivelyfiltered. In order to validate our model we introduce a new evaluationbenchmark -- HT-Step -- obtained by manually annotating a 124-hour subset ofHowTo100M\footnote{A test server is accessible at\url{https://eval.ai/web/challenges/challenge-page/2082}.} with steps sourcedfrom wikiHow articles. Experiments on this benchmark as well as zero-shotevaluations on CrossTask demonstrate that our multi-modality alignment yieldsdramatic gains over several baselines and prior works. Finally, we show thatour inner module for matching narration-to-video outperforms by a large marginthe state of the art on the HTM-Align narration-video alignment benchmark.

Learning to Ground Instructional Articles in Videos through Narrations