English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Understanding human emotions is a crucial ability for intelligent robots toprovide better human-robot interactions. The existing works are limited totrimmed video-level emotion classification, failing to locate the temporalwindow corresponding to the emotion. In this paper, we introduce a new task,named Temporal Emotion Localization in videos~(TEL), which aims to detect humanemotions and localize their corresponding temporal boundaries in untrimmedvideos with aligned subtitles. TEL presents three unique challenges compared totemporal action localization: 1) The emotions have extremely varied temporaldynamics; 2) The emotion cues are embedded in both appearances and complexplots; 3) The fine-grained temporal annotations are complicated andlabor-intensive. To address the first two challenges, we propose a noveldilated context integrated network with a coarse-fine two-stream architecture.The coarse stream captures varied temporal dynamics by modelingmulti-granularity temporal contexts. The fine stream achieves complex plotsunderstanding by reasoning the dependency between the multi-granularitytemporal contexts from the coarse stream and adaptively integrates them intofine-grained video segment features. To address the third challenge, weintroduce a cross-modal consensus learning paradigm, which leverages theinherent semantic consensus between the aligned video and subtitle to achieveweakly-supervised learning. We contribute a new testing set with 3,000manually-annotated temporal boundaries so that future research on the TELproblem can be quantitatively evaluated. Extensive experiments show theeffectiveness of our approach on temporal emotion localization. The repositoryof this work is athttps://github.com/YYJMJC/Temporal-Emotion-Localization-in-Videos.

Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos