English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Speech-driven gesture generation is highly challenging due to the randomjitters of human motion. In addition, there is an inherent asynchronousrelationship between human speech and gestures. To tackle these challenges, weintroduce a novel quantization-based and phase-guided motion-matchingframework. Specifically, we first present a gesture VQ-VAE module to learn acodebook to summarize meaningful gesture units. With each code representing aunique gesture, random jittering problems are alleviated effectively. We thenuse Levenshtein distance to align diverse gestures with different speech.Levenshtein distance based on audio quantization as a similarity metric ofcorresponding speech of gestures helps match more appropriate gestures withspeech, and solves the alignment problem of speech and gestures well. Moreover,we introduce phase to guide the optimal gesture matching based on the semanticsof context or rhythm of audio. Phase guides when text-based or speech-basedgestures should be performed to make the generated gestures more natural.Extensive experiments show that our method outperforms recent approaches onspeech-driven gesture generation. Our code, database, pre-trained models, anddemos are available at https://github.com/YoungSeng/QPGesture.

QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generation