English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A key challenge with procedure planning in instructional videos lies in howto handle a large decision space consisting of a multitude of action types thatbelong to various tasks. To understand real-world video content, an AI agentmust proficiently discern these action types (e.g., pour milk, pour water, openlid, close lid, etc.) based on brief visual observation. Moreover, it mustadeptly capture the intricate semantic relation of the action types and taskgoals, along with the variable action sequences. Recently, notable progress hasbeen made via the integration of diffusion models and visual representationlearning to address the challenge. However, existing models employ rudimentarymechanisms to utilize task information to manage the decision space. Toovercome this limitation, we introduce a simple yet effective enhancement - amasked diffusion model. The introduced mask acts akin to a task-orientedattention filter, enabling the diffusion/denoising process to concentrate on asubset of action types. Furthermore, to bolster the accuracy of taskclassification, we harness more potent visual representation learningtechniques. In particular, we learn a joint visual-text embedding, where a textembedding is generated by prompting a pre-trained vision-language model tofocus on human actions. We evaluate the method on three public datasets andachieve state-of-the-art performance on multiple metrics. Code is available athttps://github.com/ffzzy840304/Masked-PDPP.

Masked Diffusion with Task-awareness for Procedure Planning in Instructional Videos