English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Facial animation in virtual reality environments is essential forapplications that necessitate clear visibility of the user's face and theability to convey emotional signals. In our scenario, we animate the face of anoperator who controls a robotic Avatar system. The use of facial animation isparticularly valuable when the perception of interacting with a specificindividual, rather than just a robot, is intended. Purely keypoint-drivenanimation approaches struggle with the complexity of facial movements. Wepresent a hybrid method that uses both keypoints and direct visual guidancefrom a mouth camera. Our method generalizes to unseen operators and requiresonly a quick enrolment step with capture of two short videos. Multiple sourceimages are selected with the intention to cover different facial expressions.Given a mouth camera frame from the HMD, we dynamically construct the targetkeypoints and apply an attention mechanism to determine the importance of eachsource image. To resolve keypoint ambiguities and animate a broader range ofmouth expressions, we propose to inject visual mouth camera information intothe latent space. We enable training on large-scale speaking head datasets bysimulating the mouth camera input with its perspective differences and facialdeformations. Our method outperforms a baseline in quality, capability, andtemporal consistency. In addition, we highlight how the facial animationcontributed to our victory at the ANA Avatar XPRIZE Finals.

Attention-Based VR Facial Animation with Visual Mouth Camera Guidance for Immersive Telepresence Avatars