Improving Cross-Lingual Transfer Learning for End-to-End Speech Recognition with Speech Translation
원문 초록
Transfer learning from high-resource languages is known to be an efficient way to improve end-to-end automatic speech recognition (ASR) for low-resource languages. Pre-trained or jointly trained encoder-decoder models, however, do not share the language modeling (decoder) for the same language, which is likely to be inefficient for distant target languages. We introduce speech-to-text translation (ST) as an auxiliary task to incorporate additional knowledge of the target language and enable transferring from that target language. Specifically, we first translate high-resource ASR transcripts into a target low-resource language, with which a ST model is trained. Both ST and target ASR share the same attention-based encoder-decoder architecture and vocabulary. The former task then provides a fully pre-trained model for the latter, bringing up to 24.6% word error rate (WER) reduction to the baseline (direct transfer from high-resource ASR). We show that training ST with human translations is not necessary. ST trained with machine translation (MT) pseudo-labels brings consistent gains. It can even outperform those using human labels when transferred to target ASR by leveraging only 500K MT examples. Even with pseudo-labels from low-resource MT (200K examples), ST-enhanced transfer brings up to 8.9% WER reduction to direct transfer.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization
Seewo's Submission to MLC-SLM: Lessons learned from Speech Reasoning Language Models
Bi-directional Context-Enhanced Speech Large Language Models for Multilingual Conversational ASR
Mlphon: A Multifunctional Grapheme-Phoneme Conversion Tool Using Finite State Transducers
Coupled Training of Sequence-to-Sequence Models for Accented Speech Recognition
Making Language Model a Hierarchical Classifier and Generator
토론
이 논문과 연결된 포럼 글이 아직 없습니다.