English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Motion capture from a monocular video is fundamental and crucial for ushumans to naturally experience and interact with each other in Virtual Reality(VR) and Augmented Reality (AR). However, existing methods still struggle withchallenging cases involving self-occlusion and complex poses due to the lack ofeffective motion prior modeling. In this paper, we present a novel variationalmotion prior (VMP) learning approach for video-based motion capture to resolvethe above issue. Instead of directly building the correspondence between thevideo and motion domain, We propose to learn a generic latent space forcapturing the prior distribution of all natural motions, which serve as thebasis for subsequent video-based motion capture tasks. To improve thegeneralization capacity of prior space, we propose a transformer-basedvariational autoencoder pretrained over marker-based 3D mocap data, with anovel style-mapping block to boost the generation quality. Afterward, aseparate video encoder is attached to the pretrained motion generator forend-to-end fine-tuning over task-specific video datasets. Compared to existingmotion prior models, our VMP model serves as a motion rectifier that caneffectively reduce temporal jittering and failure modes in frame-wise poseestimation, leading to temporally stable and visually realistic motion captureresults. Furthermore, our VMP-based framework models motion at sequence leveland can directly generate motion clips in the forward pass, achieving real-timemotion capture during inference. Extensive experiments over both publicdatasets and in-the-wild videos have demonstrated the efficacy andgeneralization capability of our framework.

Learning Variational Motion Prior for Video-based Motion Capture