Prompting Audios Using Acoustic Properties For Emotion Representation
원문 초록
Emotions lie on a continuum, but current models treat emotions as a finite valued discrete variable. This representation does not capture the diversity in the expression of emotion. To better represent emotions we propose the use of natural language descriptions (or prompts). In this work, we address the challenge of automatically generating these prompts and training a model to better learn emotion representations from audio and prompt pairs. We use acoustic properties that are correlated to emotion like pitch, intensity, speech rate, and articulation rate to automatically generate prompts i.e. 'acoustic prompts'. We use a contrastive learning objective to map speech to their respective acoustic prompts. We evaluate our model on Emotion Audio Retrieval and Speech Emotion Recognition. Our results show that the acoustic prompts significantly improve the model's performance in EAR, in various Precision@K metrics. In SER, we observe a 3.8% relative accuracy improvement on the Ravdess dataset.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
RadiomicsRetrieval: A Customizable Framework for Medical Image Retrieval Using Radiomics Features
Dynamic Parameter Memory: Temporary LoRA-Enhanced LLM for Long-Sequence Emotion Recognition in Conversation
HapticCap: A Multimodal Dataset and Task for Understanding User Experience of Vibration Haptic Signals
How to Retrieve Examples in In-context Learning to Improve Conversational Emotion Recognition using Large Language Models?
MATER: Multi-level Acoustic and Textual Emotion Representation for Interpretable Speech Emotion Recognition
A Simple Contrastive Framework Of Item Tokenization For Generative Recommendation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.