Open AccessBrand-new Speech Animation Technology based on First Order Motion Model and MelGAN-VC
Author(s) -
Shaomin Chen,
Xinyi Gao,
Jiangtao Wang,
Yu X,
Yueling Zhang,
Gang Xu
Publication year - 2021
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1828/1/012029
Subject(s) - animation , computer science , computer facial animation , skeletal animation , computer animation , interactive skeleton driven simulation , adaptation (eye) , encoder , speech recognition , multimedia , computer graphics (images) , physics , optics , operating system
Speech animation has huge application potential in instant messaging and entertainment media fields such as videophones, virtual meetings, audio and video chats. The traditional voice-driven speech animation has the problem of a single adaptation language, and the performance-driven speech animation has the problem of high cost of capture equipment and difficult mass production. Based on the above existing problems, we propose a new method of speech animation generation, that is, given a static portrait of a person and a face-driven video, finally generate a face animation video of the character in the given portrait. The conversion system consists of two parts: face conversion and voice conversion. We noticed that the final generated face animation video has problems such as low definition, not smooth playback, and metallic sound. On this basis, this article proposes to increase the animation enhancement experiment and replace the encoder measures for improvement. Through comparative experiments, the above measures are proved to be effective.