English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose to forecast future hand-object interactions given an egocentricvideo. Instead of predicting action labels or pixels, we directly predict thehand motion trajectory and the future contact points on the next active object(i.e., interaction hotspots). This relatively low-dimensional representationprovides a concrete description of future interactions. To tackle this task, wefirst provide an automatic way to collect trajectory and hotspots labels onlarge-scale data. We then use this data to train an Object-Centric Transformer(OCT) model for prediction. Our model performs hand and object interactionreasoning via the self-attention mechanism in Transformers. OCT also provides aprobabilistic framework to sample the future trajectory and hotspots to handleuncertainty in prediction. We perform experiments on the Epic-Kitchens-55,Epic-Kitchens-100, and EGTEA Gaze+ datasets, and show that OCT significantlyoutperforms state-of-the-art approaches by a large margin. Project page isavailable at https://stevenlsw.github.io/hoi-forecast .

Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos