English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Prior works on improving speech quality with visual input typically studyeach type of auditory distortion separately (e.g., separation, inpainting,video-to-speech) and present tailored algorithms. This paper proposes to unifythese subjects and study Generalized Speech Enhancement, where the goal is notto reconstruct the exact reference clean signal, but to focus on improvingcertain aspects of speech. In particular, this paper concerns intelligibility,quality, and video synchronization. We cast the problem as audio-visual speechresynthesis, which is composed of two steps: pseudo audio-visual speechrecognition (P-AVSR) and pseudo text-to-speech synthesis (P-TTS). P-AVSR andP-TTS are connected by discrete units derived from a self-supervised speechmodel. Moreover, we utilize self-supervised audio-visual speech model toinitialize P-AVSR. The proposed model is coined ReVISE. ReVISE is the firsthigh-quality model for in-the-wild video-to-speech synthesis and achievessuperior performance on all LRS3 audio-visual enhancement tasks with a singlemodel. To demonstrates its applicability in the real world, ReVISE is alsoevaluated on EasyCom, an audio-visual benchmark collected under challengingacoustic conditions with only 1.6 hours of training data. Similarly, ReVISEgreatly suppresses noise and improves quality. Project page:https://wnhsu.github.io/ReVISE.

ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement