视频驱动人脸动画合成

罗常伟; 於俊; 汪增福

doi:10.3724/SP.J.1004.2014.02245

视频驱动人脸动画合成

doi: 10.3724/SP.J.1004.2014.02245

罗常伟¹,
於俊¹,
汪增福^1,2,3,

1.
中国科学技术大学自动化系合肥230027;
2.
中国科学院合肥智能机械研究所合肥230031;
3.
中国科学技术大学语音及语言信息处理国家工程实验室合肥230027

基金项目:

Supported by National Natural Science Foundation of China (61303150, 61472393), China Postdoctoral Science Foundation (2012M521248), and Anhui Province Innovative Funds on Intelligent Speech Technology and Industrialization (13Z02008)

计量
- 文章访问数: 2311
- HTML全文浏览量: 157
- PDF下载量: 1381
- 被引次数: 0
出版历程
- 收稿日期: 2013-06-21
- 修回日期: 2013-12-03
- 刊出日期: 2014-10-20

Synthesizing Performance-driven Facial Animation

1.
Department of Automation, University of Science and Technology of China, Hefei 230027, China;
2.
Institute of Intelligent Machines, Chinese Academy of Sciences, Hefei 230031, China;
3.
National Laboratory of Speech and Language Information Processing, University of Science and Technology of China, Hefei 230027, China

Funds:

More Information

Corresponding author: WANG Zeng-Fu Professor at Institute of Intelligent Machine, Chinese Academy of Sciences. His research interest covers computer vision, human computer interaction and intelligent robot. Corresponding author of this paper. E-mail: zfwang@ustc.edu.cn

摘要

摘要: 描述了一种实时的视频驱动的人脸动画合成系统.通过该系统,用户只要在摄像头前面表演各种脸部动作,就可以控制虚拟人脸的表情.首先,建立一个基于肌肉的三维人脸模型,并使用肌肉激励参数控制人脸形变.为提高人脸动画的真实感,将口轮匝肌分解为外圈和内圈两部分,同时建立脸部形变与下颌转动的关系.然后,使用一种实时的特征点跟踪算法跟踪视频中人脸的特征点.最后,将视频跟踪结果转换为肌肉激励参数以驱动人脸动画.实验结果表明,该系统能实时运行,合成的动画也具有较强真实感.与大部分现有的视频驱动的人脸动画方法相比,该系统不需要使用脸部标志和三维扫描设备,极大地方便了用户使用.
- 人脸动画 /
- 视频驱动 /
- 人脸跟踪 /
- 肌肉模型
Abstract: In this paper, we present a system for real-time performance-driven facial animation. With the system, the user can control the facial expression of a digital character by acting out the desired facial action in front of an ordinary camera. First, we create a muscle-based 3D face model. The muscle actuation parameters are used to animate the face model. To increase the reality of facial animation, the orbicularis oris in our face model is divided into the inner part and outer part. We also establish the relationship between jaw rotation and facial surface deformation. Second, a real-time facial tracking method is employed to track the facial features of a performer in the video. Finally, the tracked facial feature points are used to estimate muscle actuation parameters to drive the face model. Experimental results show that our system runs in real time and outputs realistic facial animations. Compared with most existing performance-based facial animation systems, ours does not require facial markers, intrusive lighting, or special scanning equipment, thus it is inexpensive and easy to use.
- Facial animation /
- performance-driven /
- face tracking /
- muscle model

HTML全文

参考文献(27)

[1]	Ersotelos N, Dong F. Building highly realistic facial modeling and animation: a survey. The Visual Computer, 2008, 24(1): 13-30
[2]	[1] Ersotelos N, Dong F. Building highly realistic facial modeling and animation: a survey. The Visual Computer, 2008, 24(1): 13-30
[3]	[2] Guenter B, Grimm C, Wood D, Malvar H, Pighin F. Making faces. In: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. New York, USA: ACM, 1998. 55-66
[4]	[3] Bradley D, Heidrich W, Popa T, Sheffer A. High resolution passive facial performance capture. ACM Transactions on Graphics, 2010, 29(4), Article No.41, DOI: 10.1145/1778765.1778778
[5]	[4] Zhang L, Snavely N, Curless B, Seitz S. Space-time faces: high rresolution capture for modeling and animation. ACM Transactions on Graphics, 2004, 23(3): 548-558
[6]	[5] Weise T, Bouaziz S, Li H, Pauly M. Realtime performance-based facial animation. ACM Transactions on Graphics, 2011, 30(4), Article No.77, DOI: 10.1145/2010324.1964972
[7]	[6] Cao C, Weng Y L, Lin S, Zhou K. 3D shape regression for real-time facial animation. ACM Transactions on Graphics, 2013,32(4), Article No.41, DOI: 10.1145/2461912.2462012
[8]	[7] Binh H, Zhu M, Deng Z. Marker optimization for facial motion acquisition and deformation. IEEE Transactions on Visualization and Computer Graphics, 2013, 19(11): 1859-1870
[9]	[8] Choe B, Lee H, Ko H S. Performance-driven muscle-based facial animation. The Journal of Visualization and Computer Animation, 2001, 12(2): 67-79
[10]	[9] Bouaziz S, Wang Y Q, Pauly M. Online modeling for realtime facial animation. ACM Transactions on Graphics, 2013, 32(4), Article No.40, DOI: 10.1145/2461912.2461976
[11]	Zeng M, Liang L, Liu X G, Bao H J. Video-driven state-aware facial animation. Computer Animation and Virtual Worlds, 2012,23(3-4): 167-178
[12]	Chai J X, Xiao J, Hodgins J. Vision-based control of 3D facial animation. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Aire-la-Ville, Switzerland: Eurographics Association, 2003. 193-206
[13]	Milborrow S, Nicolls F. Locating facial features with an extended active shape model. In: Proceedings of the 10th European Conference on Computer Vision: Part IV. Berlin, Germany: Springer-Verlag, 2008. 504-513
[14]	Huang Chen, Ding Xiao-Qing, Fang Chi. A robust and efficient facial feature tracking algorithm. Acta Automatica Sinica, 2012, 38(5): 788-796 (in Chinese)
[15]	Cao X D, Wei Y C, Wen F, Sun J. Face alignment by explicit shape regression. In: Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2012. 2887-2894
[16]	Summer R, Popovic J. Deformation transfer for triangle meshes. ACM Transactions on Graphics, 2004, 22(3): 399-405
[17]	Li Xu-Dong, Zhang Zhen-Yue. Morphable linear fitting method for facial expression synthesis. Acta Automatica Sinica, 2008, 34(5): 593-797 (in Chinese)
[18]	Vlasic D, Brand M, Pfister H, Popović J. Face transfer with multilinear models. ACM Transactions on Graphics, 2005, 24(3): 426-433
[19]	Sifakis E, Igor N, Ronald F. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Transactions on Graphics, 2005, 24(3): 417-425
[20]	Zhang Z Y, Liu Z C, Dennis A, Cohen M F, Hanson E, Shan Y. Robust and rapid generation of animated faces from video images: a model-based modeling approach. International Journal of Computer Vision, 2004, 58(1): 93-119
[21]	Waters K. A muscle model for animating three dimensional facial expression. Computer Graphics, 1987, 22(4): 17-24
[22]	Saragih J, Lucey S, Cohn J F. Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision, 2011, 91(2): 200-215
[23]	Gross R, Matthews I, Cohn J, Kanade T, Baker S. Multi-pie. Image and Vision Computing, 2010, 28(5): 807-813
[24]	Baltrusaitis T, Robinson P, Morency L. 3D constrained local model for rigid and non-rigid facial tracking. In: Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2012. 2610-2617
[25]	Baker S, Matthews I. Lucas-kanade 20 years on: a unifying framework. International Journal of Computer Vision, 2004, 56(3): 221-255
[26]	Dementhon D F, Davis L S. Model-based object pose in 25 lines of code. International Journal of Computer Vision, 1995, 15(1): 123-141
[27]	Yu Jun, Wang Zeng-Fu. 2D-3D facial video coding/decoding at ultra-low bit-rate. Acta Electronica Sinica, 2013, 41(1): 185-192 (in Chinese)