A multimodal fusion classifier is presented based on neural networks （NNs） learned with hints for automatic spontaneous affect recognition. In case that different channels can provide com- plementary information, fe...A multimodal fusion classifier is presented based on neural networks （NNs） learned with hints for automatic spontaneous affect recognition. In case that different channels can provide com- plementary information, features are utilized from four behavioral cues： frontal-view facial expres- sion, profile-view facial expression, shoulder movement, and vocalization （audio）. NNs are used in both single cue processing and multimodal fusion. Coarse categories and quadrants in the activation- evaluation dimensional space are utilized respectively as the heuristic information （hints） of NNs during training, aiming at recognition of basic emotions. With the aid of hints, the weights in NNs could learn optimal feature groupings and the subtlety and complexity of spontaneous affective states could be better modeled. The proposed method requires low computation effort and reaches high recognition accuracy, even if the training data is insufficient. Experiment results on the Semaine nat- uralistic dataset demonstrate that our method is effective and promising.展开更多
Image fusion refers to extracting meaningful information from images of different sources or modalities,and then fusing them to generate more informative images that are beneficial for subsequent applications.In recen...Image fusion refers to extracting meaningful information from images of different sources or modalities,and then fusing them to generate more informative images that are beneficial for subsequent applications.In recent years,the growing data and computing resources have promoted the development of deep learning,and image fusion technology has continued to spawn new deep learning fusion methods based on traditional fusion methods.However,high-speed railroads,as an important part of life,have their unique industry characteristics of image data,which leads to different image fusion techniques with different fusion effects in high-speed railway scenes.This research work first introduces the mainstream technology classification of image fusion,further describes the downstream tasks that image fusion techniques may combine within high-speed railway scenes,and introduces the evaluation metrics of image fusion,followed by a series of subjective and objective experiments to completely evaluate the performance level of each image fusion method in different traffic scenes,and finally provides some possible future image fusion in the field of rail transportation of research.展开更多
基金Supported by the National Natural Science Foundation of China(60905006)the Basic Research Fund of Beijing Institute ofTechnology(20120842006)
文摘A multimodal fusion classifier is presented based on neural networks （NNs） learned with hints for automatic spontaneous affect recognition. In case that different channels can provide com- plementary information, features are utilized from four behavioral cues： frontal-view facial expres- sion, profile-view facial expression, shoulder movement, and vocalization （audio）. NNs are used in both single cue processing and multimodal fusion. Coarse categories and quadrants in the activation- evaluation dimensional space are utilized respectively as the heuristic information （hints） of NNs during training, aiming at recognition of basic emotions. With the aid of hints, the weights in NNs could learn optimal feature groupings and the subtlety and complexity of spontaneous affective states could be better modeled. The proposed method requires low computation effort and reaches high recognition accuracy, even if the training data is insufficient. Experiment results on the Semaine nat- uralistic dataset demonstrate that our method is effective and promising.
基金supported in part by the National Key Research and Development Program of China,under Grant 2020YFB2103800.
文摘Image fusion refers to extracting meaningful information from images of different sources or modalities,and then fusing them to generate more informative images that are beneficial for subsequent applications.In recent years,the growing data and computing resources have promoted the development of deep learning,and image fusion technology has continued to spawn new deep learning fusion methods based on traditional fusion methods.However,high-speed railroads,as an important part of life,have their unique industry characteristics of image data,which leads to different image fusion techniques with different fusion effects in high-speed railway scenes.This research work first introduces the mainstream technology classification of image fusion,further describes the downstream tasks that image fusion techniques may combine within high-speed railway scenes,and introduces the evaluation metrics of image fusion,followed by a series of subjective and objective experiments to completely evaluate the performance level of each image fusion method in different traffic scenes,and finally provides some possible future image fusion in the field of rail transportation of research.
