Efficient Domain Adaptation for Speech Foundation Models
원문 초록
Foundation models (FMs), that are trained on broad data at scale and are adaptable to a wide range of downstream tasks, have brought large interest in the research community. Benefiting from the diverse data sources such as different modalities, languages and application domains, foundation models have demonstrated strong generalization and knowledge transfer capabilities. In this paper, we present a pioneering study towards building an efficient solution for FM-based speech recognition systems. We adopt the recently developed self-supervised BEST-RQ for pretraining, and propose the joint finetuning with both source and unsupervised target domain data using JUST Hydra. The FM encoder adapter and decoder are then finetuned to the target domain with a small amount of supervised in-domain data. On a large-scale YouTube and Voice Search task, our method is shown to be both data and model parameter efficient. It achieves the same quality with only 21.6M supervised in-domain data and 130.8M finetuned parameters, compared to the 731.1M model trained from scratch on additional 300M supervised in-domain data.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
OpusLM: A Family of Open Unified Speech Language Models
Fine-Tuning Self-Supervised Learning Models for End-to-End Pronunciation Scoring
TASK AWARE MULTI-TASK LEARNING FOR SPEECH TO TEXT TASKS
Coupled Training of Sequence-to-Sequence Models for Accented Speech Recognition
The Speechtransformer for Large-scale Mandarin Chinese Speech Recognition
First Steps Towards Voice Anonymization for Code-Switching Speech
토론
이 논문과 연결된 포럼 글이 아직 없습니다.