USTED: Improving ASR with a Unified Speech and Text Encoder-Decoder
원문 초록
Improving end-to-end speech recognition by incorporating external text data has been a longstanding research topic. There has been a recent focus on training E2E ASR models that get the performance benefits of external text data without incurring the extra cost of evaluating an external language model at inference time. In this work, we propose training ASR model jointly with a set of text-to-text auxiliary tasks with which it shares a decoder and parts of the encoder. When we jointly train ASR and masked language model with the 960-hour Librispeech and Opensubtitles data respectively, we observe WER reductions of 16% and 20% on test-other and test-clean respectively over an ASR-only baseline without any extra cost at inference time, and reductions of 6% and 8% compared to a stronger MUTE-L baseline which trains the decoder with the same text data as our model. We achieve further improvements when we train masked language model on Librispeech data or when we use machine translation as the auxiliary task, without significantly sacrificing performance on the task itself.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Making Language Model a Hierarchical Classifier and Generator
Pre-Trained LLM is a Semantic-Aware and Generalizable Segmentation Booster
LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization
Thinking in Directivity: Speech Large Language Model for Multi-Talker Directional Speech Recognition
Mlphon: A Multifunctional Grapheme-Phoneme Conversion Tool Using Finite State Transducers
Towards End-To-End Speech Recognition with Recurrent Neural Networks
토론
이 논문과 연결된 포럼 글이 아직 없습니다.