Premium
ExpressGesture: Expressive gesture generation from speech through database matching
Author(s) -
Ferstl Ylva,
Neff Michael,
McDonnell Rachel
Publication year - 2021
Publication title -
computer animation and virtual worlds
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.225
H-Index - 49
eISSN - 1546-427X
pISSN - 1546-4261
DOI - 10.1002/cav.2016
Subject(s) - gesture , computer science , speech recognition , gesture recognition , motion capture , artificial intelligence , motion (physics)
Co‐speech gestures are a vital ingredient in making virtual agents more human‐like and engaging. Automatically generated gestures based on speech‐input often lack realistic and defined gesture form. We present a database‐driven approach guaranteeing defined gesture form. We built a large corpus of over 23,000 motion‐captured co‐speech gestures and select individual gestures based on expressive gesture characteristics that can be estimated from speech audio. The expressive parameters are gesture velocity and acceleration, gesture size, arm swivel, and finger extension. Individual, parameter‐matched gestures are then combined into animated sequences. We evaluate our gesture generation system in two perceptual studies. The first study compares our method to the ground truth gestures as well as mismatched gestures. The second study compares our method to five current generative machine learning models. Our method outperformed mismatched gesture selection in the first study and showed competitive performance in the second.