As a key link in human-computer interaction,emotion recognition can enable robots to correctly perceive user emotions and provide dynamic and adjustable services according to the emotional needs of different users,whi...As a key link in human-computer interaction,emotion recognition can enable robots to correctly perceive user emotions and provide dynamic and adjustable services according to the emotional needs of different users,which is the key to improve the cognitive level of robot service.Emotion recognition based on facial expression and electrocardiogram has numerous industrial applications.First,three-dimensional convolutional neural network deep learning architecture is utilized to extract the spatial and temporal features from facial expression video data and electrocardiogram(ECG)data,and emotion classification is carried out.Then two modalities are fused in the data level and the decision level,respectively,and the emotion recognition results are then given.Finally,the emotion recognition results of single-modality and multi-modality are compared and analyzed.Through the comparative analysis of the experimental results of single-modality and multi-modality under the two fusion methods,it is concluded that the accuracy rate of multi-modal emotion recognition is greatly improved compared with that of single-modal emotion recognition,and decision-level fusion is easier to operate and more effective than data-level fusion.展开更多
为解决现有图像仿真中动漫风格迁移网络存在图像失真和风格单一等问题,提出了适用于动漫人脸风格迁移和编辑的TGFE-TrebleStyleGAN(text-guided facial editing with TrebleStyleGAN)网络框架。利用潜在空间的向量引导生成人脸图像,并在...为解决现有图像仿真中动漫风格迁移网络存在图像失真和风格单一等问题,提出了适用于动漫人脸风格迁移和编辑的TGFE-TrebleStyleGAN(text-guided facial editing with TrebleStyleGAN)网络框架。利用潜在空间的向量引导生成人脸图像,并在TrebleStyleGAN中设计了细节控制模块和特征控制模块来约束生成图像的外观。迁移网络生成的图像不仅用作风格控制信号,还用作约束细粒度分割后的编辑区域。引入文本生成图像技术,捕捉风格迁移图像和语义信息的关联性。通过在开源数据集和自建配对标签的动漫人脸数据集上的实验表明:相较于基线模型DualStyleGAN,该模型的FID降低了2.819,SSIM与NIMA分别提升了0.028和0.074。集成风格迁移与编辑的方法能够确保在生成过程中既保留原有动漫人脸细节风格,又具备灵活的编辑能力,减少了图像的失真问题,在生成图像特征的一致性和动漫人脸图像风格相似性中表现更优。展开更多
基金Supported by National Natural Science Foundation of China（61303150,61472393） China Postdoctoral Science Foundation（2012M521248） Anhui Province Innovative Funds on Intelligent Speech Technology and Industrialization（13Z02008）
基金supported by the Open Funding Project of National Key Laboratory of Human Factors Engineering(Grant NO.6142222190309)。
文摘As a key link in human-computer interaction,emotion recognition can enable robots to correctly perceive user emotions and provide dynamic and adjustable services according to the emotional needs of different users,which is the key to improve the cognitive level of robot service.Emotion recognition based on facial expression and electrocardiogram has numerous industrial applications.First,three-dimensional convolutional neural network deep learning architecture is utilized to extract the spatial and temporal features from facial expression video data and electrocardiogram(ECG)data,and emotion classification is carried out.Then two modalities are fused in the data level and the decision level,respectively,and the emotion recognition results are then given.Finally,the emotion recognition results of single-modality and multi-modality are compared and analyzed.Through the comparative analysis of the experimental results of single-modality and multi-modality under the two fusion methods,it is concluded that the accuracy rate of multi-modal emotion recognition is greatly improved compared with that of single-modal emotion recognition,and decision-level fusion is easier to operate and more effective than data-level fusion.
文摘为解决现有图像仿真中动漫风格迁移网络存在图像失真和风格单一等问题,提出了适用于动漫人脸风格迁移和编辑的TGFE-TrebleStyleGAN(text-guided facial editing with TrebleStyleGAN)网络框架。利用潜在空间的向量引导生成人脸图像,并在TrebleStyleGAN中设计了细节控制模块和特征控制模块来约束生成图像的外观。迁移网络生成的图像不仅用作风格控制信号,还用作约束细粒度分割后的编辑区域。引入文本生成图像技术,捕捉风格迁移图像和语义信息的关联性。通过在开源数据集和自建配对标签的动漫人脸数据集上的实验表明:相较于基线模型DualStyleGAN,该模型的FID降低了2.819,SSIM与NIMA分别提升了0.028和0.074。集成风格迁移与编辑的方法能够确保在生成过程中既保留原有动漫人脸细节风格,又具备灵活的编辑能力,减少了图像的失真问题,在生成图像特征的一致性和动漫人脸图像风格相似性中表现更优。
