English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Though significant progress has been made for speaker-dependentVideo-to-Speech (VTS) synthesis, little attention is devoted to multi-speakerVTS that can map silent video to speech, while allowing flexible control ofspeaker identity, all in a single system. This paper proposes a novelmulti-speaker VTS system based on cross-modal knowledge transfer from voiceconversion (VC), where vector quantization with contrastive predictive coding(VQCPC) is used for the content encoder of VC to derive discrete phoneme-likeacoustic units, which are transferred to a Lip-to-Index (Lip2Ind) network toinfer the index sequence of acoustic units. The Lip2Ind network can thensubstitute the content encoder of VC to form a multi-speaker VTS system toconvert silent video to acoustic units for reconstructing accurate spokencontent. The VTS system also inherits the advantages of VC by using a speakerencoder to produce speaker representations to effectively control the speakeridentity of generated speech. Extensive evaluations verify the effectiveness ofproposed approach, which can be applied in both constrained vocabulary and openvocabulary conditions, achieving state-of-the-art performance in generatinghigh-quality speech with high naturalness, intelligibility and speakersimilarity. Our demo page is released here:https://wendison.github.io/VCVTS-demo/

VCVTS: Multi-speaker Video-to-Speech synthesis via cross-modal knowledge transfer from voice conversion