English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Egocentric sensors such as AR/VR devices capture human-object interactionsand offer the potential to provide task-assistance by recalling 3D locations ofobjects of interest in the surrounding environment. This capability requiresinstance tracking in real-world 3D scenes from egocentric videos (IT3DEgo). Weexplore this problem by first introducing a new benchmark dataset, consistingof RGB and depth videos, per-frame camera pose, and instance-level annotationsin both 2D camera and 3D world coordinates. We present an evaluation protocolwhich evaluates tracking performance in 3D coordinates with two settings forenrolling instances to track: (1) single-view online enrollment where aninstance is specified on-the-fly based on the human wearer's interactions. and(2) multi-view pre-enrollment where images of an instance to be tracked arestored in memory ahead of time. To address IT3DEgo, we first re-purpose methodsfrom relevant areas, e.g., single object tracking (SOT) -- running SOT methodsto track instances in 2D frames and lifting them to 3D using camera pose anddepth. We also present a simple method that leverages pretrained segmentationand detection models to generate proposals from RGB frames and match proposalswith enrolled instance images. Perhaps surprisingly, our extensive experimentsshow that our method (with no finetuning) significantly outperforms SOT-basedapproaches. We conclude by arguing that the problem of egocentric instancetracking is made easier by leveraging camera pose and using a 3D allocentric(world) coordinate representation.

Instance Tracking in 3D Scenes from Egocentric Videos