Artificial intelligence,such as deep learning technology,has advanced the study of facial expression recognition since facial expression carries rich emotional information and is significant for many naturalistic situ...Artificial intelligence,such as deep learning technology,has advanced the study of facial expression recognition since facial expression carries rich emotional information and is significant for many naturalistic situations.To pursue a high facial expression recognition accuracy,the network model of deep learning is generally designed to be very deep while the model’s real-time performance is typically constrained and limited.With MobileNetV3,a lightweight model with a good accuracy,a further study is conducted by adding a basic ResNet module to each of its existing modules and an SSH(Single Stage Headless Face Detector)context module to expand the model’s perceptual field.In this article,the enhanced model named Res-MobileNetV3,could alleviate the subpar of real-time performance and compress the size of large network models,which can process information at a rate of up to 33 frames per second.Although the improved model has been verified to be slightly inferior to the current state-of-the-art method in aspect of accuracy rate on the publically available face expression datasets,it can bring a good balance on accuracy,real-time performance,model size and model complexity in practical applications.展开更多
Background The impairment of facial expression recognition has become a biomarker for early identification of first-episode schizophrenia, and this kind of research is increasing.Aims To explore the differences in bra...Background The impairment of facial expression recognition has become a biomarker for early identification of first-episode schizophrenia, and this kind of research is increasing.Aims To explore the differences in brain area activation using different degrees of disgusted facial expression recognition in antipsychotic-na?ve patients with firstepisode schizophrenia and healthy controls.Methods In this study, facial expression recognition tests were performed on 30 first-episode, antipsychoticna?ve patients with schizophrenia(treatment group) and 30 healthy subjects(control group) with matched age, educational attainment and gender. Functional MRI was used for comparing the differences of the brain areas of activation between the two groups.Results The average response time difference between the patient group and the control group in the ‘high degree of disgust' facial expression recognition task was statistically significant(1.359(0.408)/2.193(0.625), F=26.65, p<0.001), and the correct recognition rate of the treatment group was lower than that of the control group(41.05(22.25)/59.84(13.91, F=19.81, p<0.001). Compared with the control group, the left thalamus, right lingual gyrus and right middle temporal gyrus were negatively activated in the patients with first-episode schizophrenia in the ‘high degree of disgust' emotion recognition, and there was a significant activation in the left and right middle temporal gyrus and the right caudate nucleus. However, there was no significant activation difference in the ‘low degree of disgust' recognition.Conclusions In patients with first-episode schizophrenia, the areas of facial recognition impairment are significantly different in different degrees of disgust facial expression recognition.展开更多
A new algorithm taking the spatial context of local features into account by utilizing contextualized histograms was proposed to recognize facial expression. The contextualized histograms were extracted fromtwo widely...A new algorithm taking the spatial context of local features into account by utilizing contextualized histograms was proposed to recognize facial expression. The contextualized histograms were extracted fromtwo widely used descriptors—the local binary pattern（ LBP） and weber local descriptor（ WLD）. The LBP and WLD feature histograms were extracted separately fromeach facial image,and contextualized histogram was generated as feature vectors to feed the classifier. In addition,the human face was divided into sub-blocks and each sub-block was assigned different weights by their different contributions to the intensity of facial expressions to improve the recognition rate. With the support vector machine（SVM） as classifier,the experimental results on the 2D texture images fromthe 3D-BU FE dataset indicated that contextualized histograms improved facial expression recognition performance when local features were employed.展开更多
细粒度表情识别任务因其包含更丰富真实的人类情感而备受关注.现有面部表情识别算法通过提取局部关键区域等方式学习更优的图像表征.然而,这些方法忽略了图像数据集内在的结构关系,且没有充分利用标签间的语义关联度以及图像和标签间的...细粒度表情识别任务因其包含更丰富真实的人类情感而备受关注.现有面部表情识别算法通过提取局部关键区域等方式学习更优的图像表征.然而,这些方法忽略了图像数据集内在的结构关系,且没有充分利用标签间的语义关联度以及图像和标签间的相关性,导致所学特征带来的性能提升有限.其次,现有细粒度表情识别方法并未有效利用和挖掘粗细粒度的层级关系,因而限制了模型的识别性能.此外,现有细粒度表情识别算法忽略了由于标注主观性和情感复杂性导致的标签歧义性问题,极大影响了模型的识别性能.针对上述问题,本文提出一种基于关系感知和标签消歧的细粒度面部表情识别算法(fine-grained facial expression recognition algorithm based on Relationship-Awareness and Label Disambiguation,RALD).该算法通过构建层级感知的图像特征增强网络,充分挖掘图像之间、层级标签之间以及图像和标签之间的依赖关系,以获得更具辨别性的图像特征.针对标签歧义性问题,算法设计了基于近邻样本的标签分布学习模块,通过整合邻域信息进行标签消歧,进一步提升模型识别性能.在细粒度表情识别数据集FG-Emotions上算法的准确度达到97.34%,在粗粒度表情识别数据集RAF-DB上比现有主流表情分类方法提高了0.80%~4.55%.展开更多
基金supported by China Academy of Railway Sciences Corporation Limited(No.2021YJ127).
文摘Artificial intelligence,such as deep learning technology,has advanced the study of facial expression recognition since facial expression carries rich emotional information and is significant for many naturalistic situations.To pursue a high facial expression recognition accuracy,the network model of deep learning is generally designed to be very deep while the model’s real-time performance is typically constrained and limited.With MobileNetV3,a lightweight model with a good accuracy,a further study is conducted by adding a basic ResNet module to each of its existing modules and an SSH(Single Stage Headless Face Detector)context module to expand the model’s perceptual field.In this article,the enhanced model named Res-MobileNetV3,could alleviate the subpar of real-time performance and compress the size of large network models,which can process information at a rate of up to 33 frames per second.Although the improved model has been verified to be slightly inferior to the current state-of-the-art method in aspect of accuracy rate on the publically available face expression datasets,it can bring a good balance on accuracy,real-time performance,model size and model complexity in practical applications.
基金Shanghai Mental Health Center hospital-level research project(2016-YJ-04)National Key Technology R&D Program of China during the 10th Five-Year Plan Period(2007BAI17B04)+2 种基金National Key Research and Development Program(2016YFC1306805)National Natural Science Foundation of China(81471359)Shanghai Municipal Committee of Science and Technology Guide Project of Chinese and Western Medicine(14411963400)
文摘Background The impairment of facial expression recognition has become a biomarker for early identification of first-episode schizophrenia, and this kind of research is increasing.Aims To explore the differences in brain area activation using different degrees of disgusted facial expression recognition in antipsychotic-na?ve patients with firstepisode schizophrenia and healthy controls.Methods In this study, facial expression recognition tests were performed on 30 first-episode, antipsychoticna?ve patients with schizophrenia(treatment group) and 30 healthy subjects(control group) with matched age, educational attainment and gender. Functional MRI was used for comparing the differences of the brain areas of activation between the two groups.Results The average response time difference between the patient group and the control group in the ‘high degree of disgust' facial expression recognition task was statistically significant(1.359(0.408)/2.193(0.625), F=26.65, p<0.001), and the correct recognition rate of the treatment group was lower than that of the control group(41.05(22.25)/59.84(13.91, F=19.81, p<0.001). Compared with the control group, the left thalamus, right lingual gyrus and right middle temporal gyrus were negatively activated in the patients with first-episode schizophrenia in the ‘high degree of disgust' emotion recognition, and there was a significant activation in the left and right middle temporal gyrus and the right caudate nucleus. However, there was no significant activation difference in the ‘low degree of disgust' recognition.Conclusions In patients with first-episode schizophrenia, the areas of facial recognition impairment are significantly different in different degrees of disgust facial expression recognition.
基金Supported by the National Natural Science Foundation of China(60772066)
文摘A new algorithm taking the spatial context of local features into account by utilizing contextualized histograms was proposed to recognize facial expression. The contextualized histograms were extracted fromtwo widely used descriptors—the local binary pattern（ LBP） and weber local descriptor（ WLD）. The LBP and WLD feature histograms were extracted separately fromeach facial image,and contextualized histogram was generated as feature vectors to feed the classifier. In addition,the human face was divided into sub-blocks and each sub-block was assigned different weights by their different contributions to the intensity of facial expressions to improve the recognition rate. With the support vector machine（SVM） as classifier,the experimental results on the 2D texture images fromthe 3D-BU FE dataset indicated that contextualized histograms improved facial expression recognition performance when local features were employed.
文摘细粒度表情识别任务因其包含更丰富真实的人类情感而备受关注.现有面部表情识别算法通过提取局部关键区域等方式学习更优的图像表征.然而,这些方法忽略了图像数据集内在的结构关系,且没有充分利用标签间的语义关联度以及图像和标签间的相关性,导致所学特征带来的性能提升有限.其次,现有细粒度表情识别方法并未有效利用和挖掘粗细粒度的层级关系,因而限制了模型的识别性能.此外,现有细粒度表情识别算法忽略了由于标注主观性和情感复杂性导致的标签歧义性问题,极大影响了模型的识别性能.针对上述问题,本文提出一种基于关系感知和标签消歧的细粒度面部表情识别算法(fine-grained facial expression recognition algorithm based on Relationship-Awareness and Label Disambiguation,RALD).该算法通过构建层级感知的图像特征增强网络,充分挖掘图像之间、层级标签之间以及图像和标签之间的依赖关系,以获得更具辨别性的图像特征.针对标签歧义性问题,算法设计了基于近邻样本的标签分布学习模块,通过整合邻域信息进行标签消歧,进一步提升模型识别性能.在细粒度表情识别数据集FG-Emotions上算法的准确度达到97.34%,在粗粒度表情识别数据集RAF-DB上比现有主流表情分类方法提高了0.80%~4.55%.
