Multi-Modal Emotion Recognition by Text, Speech and Video Using Pretrained Transformers
원문 초록
Due to the complex nature of human emotions and the diversity of emotion representation methods in humans, emotion recognition is a challenging field. In this research, three input modalities, namely text, audio (speech), and video, are employed to generate multimodal feature vectors. For generating features for each of these modalities, pre-trained Transformer models with fine-tuning are utilized. In each modality, a Transformer model is used with transfer learning to extract feature and emotional structure. These features are then fused together, and emotion recognition is performed using a classifier. To select an appropriate fusion method and classifier, various feature-level and decision-level fusion techniques have been experimented with, and ultimately, the best model, which combines feature-level fusion by concatenating feature vectors and classification using a Support Vector Machine on the IEMOCAP multimodal dataset, achieves an accuracy of 75.42%. Keywords: Multimodal Emotion Recognition, IEMOCAP, Self-Supervised Learning, Transfer Learning, Transformer.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Multimodal Emotion Recognition with Transformer-Based Self Supervised Feature Fusion
Benchmarking histopathology foundation models in a multi-center dataset for skin cancer subtyping
Towards Robust Multimodal Emotion Recognition under Missing Modalities and Distribution Shifts
A Robust Incomplete Multimodal Low-Rank Adaptation Approach for Emotion Recognition
GSDNet: Revisiting Incomplete Multimodal-Diffusion from Graph Spectrum Perspective for Conversation Emotion Recognition
EfficientFER: EfficientNetv2 Based Deep Learning Approach for Facial Expression Recognition
토론
이 논문과 연결된 포럼 글이 아직 없습니다.