English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recently audio-driven talking face video generation has attractedconsiderable attention. However, very few researches address the issue ofemotional editing of these talking face videos with continuously controllableexpressions, which is a strong demand in the industry. The challenge is thatspeech-related expressions and emotion-related expressions are often highlycoupled. Meanwhile, traditional image-to-image translation methods cannot workwell in our application due to the coupling of expressions with otherattributes such as poses, i.e., translating the expression of the character ineach frame may simultaneously change the head pose due to the bias of thetraining data distribution. In this paper, we propose a high-quality facialexpression editing method for talking face videos, allowing the user to controlthe target emotion in the edited video continuously. We present a newperspective for this task as a special case of motion information editing,where we use a 3DMM to capture major facial movements and an associated texturemap modeled by a StyleGAN to capture appearance details. Both representations(3DMM and texture map) contain emotional information and can be continuouslymodified by neural networks and easily smoothed by averaging incoefficient/latent spaces, making our method simple yet effective. We alsointroduce a mouth shape preservation loss to control the trade-off between lipsynchronization and the degree of exaggeration of the edited expression.Extensive experiments and a user study show that our method achievesstate-of-the-art performance across various evaluation criteria.

Continuously Controllable Facial Expression Editing in Talking Face Videos