English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this work, we present a multimodal solution to the problem of 4D facereconstruction from monocular videos. 3D face reconstruction from 2D images isan under-constrained problem due to the ambiguity of depth. State-of-the-artmethods try to solve this problem by leveraging visual information from asingle image or video, whereas 3D mesh animation approaches rely more on audio.However, in most cases (e.g. AR/VR applications), videos include both visualand speech information. We propose AVFace that incorporates both modalities andaccurately reconstructs the 4D facial and lip motion of any speaker, withoutrequiring any 3D ground truth for training. A coarse stage estimates theper-frame parameters of a 3D morphable model, followed by a lip refinement, andthen a fine stage recovers facial geometric details. Due to the temporal audioand video information captured by transformer-based modules, our method isrobust in cases when either modality is insufficient (e.g. face occlusions).Extensive qualitative and quantitative evaluation demonstrates the superiorityof our method over the current state-of-the-art.

AVFace: Towards Detailed Audio-Visual 4D Face Reconstruction