Premium
Synthesizing multimodal utterances for conversational agents
Author(s) -
Kopp Stefan,
Wachsmuth Ipke
Publication year - 2004
Publication title -
computer animation and virtual worlds
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.225
H-Index - 49
eISSN - 1546-427X
pISSN - 1546-4261
DOI - 10.1002/cav.6
Subject(s) - computer science , gesture , focus (optics) , modalities , articulation (sociology) , speech recognition , coarticulation , adaptation (eye) , natural (archaeology) , concatenation (mathematics) , human–computer interaction , natural language processing , artificial intelligence , psychology , neuroscience , social science , physics , mathematics , archaeology , vowel , combinatorics , sociology , politics , law , political science , optics , history
Conversational agents are supposed to combine speech with non‐verbal modalities for intelligible multimodal utterances. In this paper, we focus on the generation of gesture and speech from XML‐based descriptions of their overt form. An incremental production model is presented that combines the synthesis of synchronized gestural, verbal, and facial behaviors with mechanisms for linking them in fluent utterances with natural co‐articulation and transition effects. In particular, an efficient kinematic approach for animating hand gestures from shape specifications is presented, which provides fine adaptation to temporal constraints that are imposed by cross‐modal synchrony. Copyright © 2004 John Wiley & Sons, Ltd.