• 首页
  • 期刊简介
  • 编委会
  • 投稿指南
  • 收录情况
  • 杂志订阅
  • 联系我们
引用本文:豆子闻,李文书.基于Transformer的面部动画生成[J].软件工程,2023,26(12):59-62.【点击复制】
【打印本页】   【下载PDF全文】   【查看/发表评论】  【下载PDF阅读器】  
←前一篇|后一篇→ 过刊浏览
分享到: 微信 更多
基于Transformer的面部动画生成
豆子闻, 李文书
(浙江理工大学计算机科学与技术学院, 浙江 杭州 310018)
1055459759@qq.com; charlie@zstu.edu.cn
摘 要: 在面部动画生成领域,克服人脸几何形状的复杂性是一项极具挑战性的任务。为了更好地应对这一挑战,文章采用了一种创新的方法,即将经过一维卷积堆叠和自注意力提取后的音频特征作为输入,通过Transformer模型从音频信号中生成面部动画。这个过程采用时间自回归模型逐步合成面部运动。使用BIWI数据集开展实验证明,该方法成功地将唇部顶点误差率缩小至令人满意的6.123%,同步率超过MeshTalk79.64%,这意味该方法在口型同步和面部表情生成方面表现出色,在完成面部动画生成任务中表现出很高的潜力,可为未来相关研究提供方向和参考。
关键词: 动画生成;自回归;深度学习;唇形同步;Transformer
中图分类号: TP391    文献标识码: A
Facial Animation Generation Based on Transformer
DOU Ziwen, LI Wenshu
(School of Computer Science and Technology, Zhejiang Sci-Tech University, Hangzhou 310018, China)
1055459759@qq.com; charlie@zstu.edu.cn
Abstract: In the field of facial animation generation, overcoming the complexity of face geometry has always been a highly challenging task. To better meet this challenge, this paper proposes an innovative approach which uses audio features that have undergone one-dimensional convolutional stacking and self-attention extraction as input, and generates facial animations from audio signals using a Transformer model. During the process, a time auto-regression model is used to gradually synthesize facial movements. Experiments on BIWI dataset show that this method has successfully reduced the lip vertex error rate to a satisfactory 6.123% , with a synchronization rate 79.64% higher than MeshTalk. This means that the proposed method performs well in lip synchronization and facial expression generation, and shows high potential in accomplishing facial animation generation tasks. It provides directions and reference for future related research
Keywords: animation generation; auto-regression; deep learning; lip synchronization; Transformer


版权所有:软件工程杂志社
地址:辽宁省沈阳市浑南区新秀街2号 邮政编码:110179
电话:0411-84767887 传真:0411-84835089 Email:semagazine@neusoft.edu.cn
备案号:辽ICP备17007376号-1
技术支持:北京勤云科技发展有限公司

用微信扫一扫

用微信扫一扫