English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many online action prediction models observe complete frames to locate andattend to informative subregions in the frames called glimpses and recognize anongoing action based on global and local information. However, in applicationswith constrained resources, an agent may not be able to observe the completeframe, yet must still locate useful glimpses to predict an incomplete actionbased on local information only. In this paper, we develop Glimpse Transformers(GliTr), which observe only narrow glimpses at all times, thus predicting anongoing action and the following most informative glimpse location based on thepartial spatiotemporal information collected so far. In the absence of a groundtruth for the optimal glimpse locations for action recognition, we train GliTrusing a novel spatiotemporal consistency objective: We require GliTr to attendto the glimpses with features similar to the corresponding complete frames(i.e. spatial consistency) and the resultant class logits at time $t$equivalent to the ones predicted using whole frames up to $t$ (i.e. temporalconsistency). Inclusion of our proposed consistency objective yields ~10%higher accuracy on the Something-Something-v2 (SSv2) dataset than the baselinecross-entropy objective. Overall, despite observing only ~33% of the total areaper frame, GliTr achieves 53.02% and 93.91% accuracy on the SSv2 and Jesterdatasets, respectively.

GliTr: Glimpse Transformers with Spatiotemporal Consistency for Online Action Prediction