English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With an enormous number of hand images generated over time, unleashing poseknowledge from unlabeled images for supervised hand mesh estimation is anemerging yet challenging topic. To alleviate this issue, semi-supervised andself-supervised approaches have been proposed, but they are limited by thereliance on detection models or conventional ResNet backbones. In this paper,inspired by the rapid progress of Masked Image Modeling (MIM) in visualclassification tasks, we propose a novel self-supervised pre-training strategyfor regressing 3D hand mesh parameters. Our approach involves a unified andmulti-granularity strategy that includes a pseudo keypoint alignment module inthe teacher-student framework for learning pose-aware semantic class tokens.For patch tokens with detailed locality, we adopt a self-distillation mannerbetween teacher and student network based on MIM pre-training. To better fitlow-level regression tasks, we incorporate pixel reconstruction tasks formulti-level representation learning. Additionally, we design a strong poseestimation baseline using a simple vanilla vision Transformer (ViT) as thebackbone and attach a PyMAF head after tokens for regression. Extensiveexperiments demonstrate that our proposed approach, named HandMIM, achievesstrong performance on various hand mesh estimation tasks. Notably, HandMIMoutperforms specially optimized architectures, achieving 6.29mm and 8.00mmPAVPE (Vertex-Point-Error) on challenging FreiHAND and HO3Dv2 test sets,respectively, establishing new state-of-the-art records on 3D hand meshestimation.

HandMIM: Pose-Aware Self-Supervised Learning for 3D Hand Mesh Estimation