摘要
音频驱动的人脸图像生成技术旨在通过输入音频和静态图像(或视频)生成对应的动态说话视频,在虚拟角色交互、数字媒体创作、游戏开发等多个领域展现出显著的应用潜力,具有广阔的研究前景和重要的研究价值。在分类分析常用的音频特征提取方法、中间表示方法和特征融合方法的基础上,对基于生成对抗网络、神经辐射场和扩散模型的音频驱动人脸解决方案做总结概述,通过分析各方案的关键技术、对比部分方案的生成效果,归纳出各方案在生成图像质量、唇形同步性以及实时性等方面的优势与不足。辨析常用的数据集和评价指标,指出目前音频驱动人脸图像生成存在的挑战,并对未来可能的研究方向作出展望。
Voice-driven face image generation technology aims to generate the corresponding dynamic speech video by inputting audio and static image(or video).This technology shows significant application potential in many fields such as virtual role interaction,digital media creation,game development and so on,which has broad research prospects and important research value.On the basis of classifying and analyzing the commonly used audio feature extraction methods,intermediate representation methods and feature fusion methods,this paper summarizes the audio driven face solutions based on the generation countermeasure network,neural radiation field and diffusion model.By analyzing the key technologies of each scheme and comparing the generation effects of some schemes,the advantages and disadvantages of each scheme in image quality,lip synchronicity and real-time performance are summarized.This paper analyzes the commonly used data sets and evaluation indicators,points out the current challenges in voice-driven face image generation,and looks forward to the possible research directions in the future.
作者
胡原平
阎红灿
HU Yuanping;YAN Hongcan(School of Science,North China University of Science and Technology,Tangshan,Hebei 063210,China;Hebei Key Laboratory of Data Science and Application,Tangshan,Hebei 063210,China)
出处
《计算机工程与应用》
北大核心
2025年第17期33-46,共14页
Computer Engineering and Applications
基金
河北省高等教育实践项目(2023GJJG226)
华北理工大学重点项目(医工融合领域)(ZD-YG-202316)。
关键词
音频驱动
人脸图像生成
多模态
神经网络
数字人
audio driver
face image generation
multimodal
neural network
digital human
作者简介
胡原平(2000-),女,硕士研究生,CCF学生会员,研究方向为计算机视觉、图像隐写分析;通信作者:阎红灿(1969-),女,博士,教授,研究方向为智能推荐、数字图像处理、图像隐写分析。E-mail:2429074504@qq.com。
