English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Talking face video generation with arbitrary speech audio is a significantchallenge within the realm of digital human technology. The previous studieshave emphasized the significance of audio-lip synchronization and visualquality. Currently, limited attention has been given to the learning of visualuncertainty, which creates several issues in existing systems, includinginconsistent visual quality and unreliable performance across different inputconditions. To address the problem, we propose a Joint Uncertainty LearningNetwork (JULNet) for high-quality talking face video generation, whichincorporates a representation of uncertainty that is directly related to visualerror. Specifically, we first design an uncertainty module to individuallypredict the error map and uncertainty map after obtaining the generated image.The error map represents the difference between the generated image and theground truth image, while the uncertainty map is used to predict theprobability of incorrect estimates. Furthermore, to match the uncertaintydistribution with the error distribution through a KL divergence term, weintroduce a histogram technique to approximate the distributions. By jointlyoptimizing error and uncertainty, the performance and robustness of our modelcan be enhanced. Extensive experiments demonstrate that our method achievessuperior high-fidelity and audio-lip synchronization in talking face videogeneration compared to previous methods.

Audio-Driven Talking Face Video Generation with Joint Uncertainty Learning