Mixture of LoRA Experts for Low-Resourced Multi-Accent Automatic Speech Recognition
원문 초록
We aim to improve the robustness of Automatic Speech Recognition (ASR) systems against non-native speech, particularly in low-resourced multi-accent settings. We introduce Mixture of Accent-Specific LoRAs (MAS-LoRA), a fine-tuning method that leverages a mixture of Low-Rank Adaptation (LoRA) experts, each specialized in a specific accent. This method can be used when the accent is known or unknown at inference time, without the need to fine-tune the model again. Our experiments, conducted using Whisper on the L2-ARCTIC corpus, demonstrate significant improvements in Word Error Rate compared to regular LoRA and full fine-tuning when the accent is unknown. When the accent is known, the results further improve. Furthermore, MAS-LoRA shows less catastrophic forgetting than the other fine-tuning methods. To the best of our knowledge, this is the first use of a mixture of LoRA experts for non-native multi-accent ASR.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
NonverbalTTS: A Public English Corpus of Text-Aligned Nonverbal Vocalizations with Emotion Annotations for Text-to-Speech
WhisperKit: On-device Real-time ASR with Billion-Scale Transformers
Lightweight Target-Speaker-Based Overlap Transcription for Practical Streaming ASR
AI-Generated Song Detection via Lyrics Transcripts
End-to-End Spoken Grammatical Error Correction
LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization
토론
이 논문과 연결된 포럼 글이 아직 없습니다.