Real-Time Speech-Driven Face Animation With Expressions Using Neural Networks  

Oleh:

Huang, T. S. ; Wen, Zhen ; Hong, Pengyu

Jenis: Article from Journal - ilmiah internasional
Dalam koleksi: IEEE Transactions on Neural Networks vol. 13 no. 4 (2002), page 916-927.
Topik: NEURAL NETWORKS; real - time; speech - drive; face animation; expression; neural networks

Ketersediaan

Perpustakaan Pusat (Semanggi) Nomor Panggil: II36.7A Non-tandon: 1 (dapat dipinjam: 0) Tandon: tidak ada

Lihat Detail Induk

Isi artikel

A real - time speech - driven synthetic talking face provides an effective multimodal communication interface in distributed collaboration environments. Nonverbal gestures such as facial expressions are important to human communication and should be considered by speech - driven face animation systems. In this paper, we present a framework that systematically addresses facial deformation modeling, automatic facial motion analysis, and real - time speech - driven face animation with expression using neural networks. Based on this framework, we learn a quantitative visual representation of the facial deformations, called the motion units (MU s). A facial deformation can be approximated by a linear combination of the MU s weighted by MU parameters (MUP s). We develop an MU - based facial motion tracking algorithm which is used to collect an audio - visual training database. Then, we construct a real-time audio - to - MUP mapping by training a set of neural networks using the collected audio - visual training database. The quantitative evaluation of the mapping shows the effectiveness of the proposed approach. Using the proposed method, we develop the functionality of real - time speech - driven face animation with expressions for the iFACE system. Experimental results show that the synthetic expressive talking face of the iFACE system is comparable with a real face in terms of the effectiveness of their influences on bimodal human emotion perception.

Opini Anda

Klik untuk menuliskan opini Anda tentang koleksi ini!

Kembali