English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Multimodal-driven talking face generation refers to animating a portrait withthe given pose, expression, and gaze transferred from the driving image andvideo, or estimated from the text and audio. However, existing methods ignorethe potential of text modal, and their generators mainly follow thesource-oriented feature rearrange paradigm coupled with unstable GANframeworks. In this work, we first represent the emotion in the text prompt,which could inherit rich semantics from the CLIP, allowing flexible andgeneralized emotion control. We further reorganize these tasks as thetarget-oriented texture transfer and adopt the Diffusion Models. Morespecifically, given a textured face as the source and the rendered faceprojected from the desired 3DMM coefficients as the target, our proposedTexture-Geometry-aware Diffusion Model decomposes the complex transfer probleminto multi-conditional denoising process, where a Texture Attention-basedmodule accurately models the correspondences between appearance and geometrycues contained in source and target conditions, and incorporate extra implicitinformation for high-fidelity talking face generation. Additionally, TGDM canbe gracefully tailored for face swapping. We derive a novel paradigm free ofunstable seesaw-style optimization, resulting in simple, stable, and effectivetraining and inference schemes. Extensive experiments demonstrate thesuperiority of our method.

Multimodal-driven Talking Face Generation via a Unified Diffusion-based Generator