English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Temporal action proposal generation (TAPG) is a challenging task, whichrequires localizing action intervals in an untrimmed video. Intuitively, we ashumans, perceive an action through the interactions between actors, relevantobjects, and the surrounding environment. Despite the significant progress ofTAPG, a vast majority of existing methods ignore the aforementioned principleof the human perceiving process by applying a backbone network into a givenvideo as a black-box. In this paper, we propose to model these interactionswith a multi-modal representation network, namely, Actors-Objects-EnvironmentInteraction Network (AOE-Net). Our AOE-Net consists of two modules, i.e.,perception-based multi-modal representation (PMR) and boundary-matching module(BMM). Additionally, we introduce adaptive attention mechanism (AAM) in PMR tofocus only on main actors (or relevant objects) and model the relationshipsamong them. PMR module represents each video snippet by a visual-linguisticfeature, in which main actors and surrounding environment are represented byvisual information, whereas relevant objects are depicted by linguisticfeatures through an image-text model. BMM module processes the sequence ofvisual-linguistic features as its input and generates action proposals.Comprehensive experiments and extensive ablation studies on ActivityNet-1.3 andTHUMOS-14 datasets show that our proposed AOE-Net outperforms previousstate-of-the-art methods with remarkable performance and generalization forboth TAPG and temporal action detection. To prove the robustness andeffectiveness of AOE-Net, we further conduct an ablation study on egocentricvideos, i.e. EPIC-KITCHENS 100 dataset. Source code is available uponacceptance.

AOE-Net: Entities Interactions Modeling with Adaptive Attention Mechanism for Temporal Action Proposals Generation