Premium
Visual speech, a trajectory in viseme space
Author(s) -
Kalberer Gregor A.,
Müller Pascal,
Gool Luc Van
Publication year - 2003
Publication title -
international journal of imaging systems and technology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.359
H-Index - 47
eISSN - 1098-1098
pISSN - 0899-9457
DOI - 10.1002/ima.10044
Subject(s) - viseme , coarticulation , computer science , animation , copying , face (sociological concept) , computer facial animation , speech recognition , space (punctuation) , artificial intelligence , computer vision , landmark , computer graphics (images) , computer animation , speech synthesis , speech technology , vowel , political science , law , operating system , social science , sociology
Efficient, realistic face animation is still a challenge. A system is proposed that yields realistic animations for speech. It starts from real 3D face dynamics, observed at frame rate for thousands of points on the faces of speaking actors. When asked to animate a face it replicates the visemes that it has learned, and adds the necessary coarticulation effects. The speech animation could be based on as few as 16 modes, extracted through independent component analysis from the observed face dynamics. Rather than animating via verbatim copying the deformation fields that come with the different visemes are adapted to the shape of the given face. By localizing the face to be animated in a face space, where also the locations of the example faces are known, visemes are adapted automatically according to the relative distance with respect to these examples. © 2003 Wiley Periodicals, Inc. Int J Imaging Syst Technol 13: 74–84, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.10044