English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Egocentric action anticipation is a challenging task that aims to makeadvanced predictions of future actions from current and historical observationsin the first-person view. Most existing methods focus on improving the modelarchitecture and loss function based on the visual input and recurrent neuralnetwork to boost the anticipation performance. However, these methods, whichmerely consider visual information and rely on a single network architecture,gradually reach a performance plateau. In order to fully understand what hasbeen observed and capture the dependencies between current observations andfuture actions well enough, we propose a novel visual-semantic fusion enhancedand Transformer GRU-based action anticipation framework in this paper. Firstly,high-level semantic information is introduced to improve the performance ofaction anticipation for the first time. We propose to use the semantic featuresgenerated based on the class labels or directly from the visual observations toaugment the original visual features. Secondly, an effective visual-semanticfusion module is proposed to make up for the semantic gap and fully utilize thecomplementarity of different modalities. Thirdly, to take advantage of both theparallel and autoregressive models, we design a Transformer based encoder forlong-term sequential modeling and a GRU-based decoder for flexible iterationdecoding. Extensive experiments on two large-scale first-person view datasets,i.e., EPIC-Kitchens and EGTEA Gaze+, validate the effectiveness of our proposedmethod, which achieves new state-of-the-art performance, outperforming previousapproaches by a large margin.

VS-TransGRU: A Novel Transformer-GRU-based Framework Enhanced by Visual-Semantic Fusion for Egocentric Action Anticipation