PolySpeech: Exploring Unified Multitask Speech Models for Competitiveness with Single-task Models
원문 초록
Recently, there have been attempts to integrate various speech processing tasks into a unified model. However, few previous works directly demonstrated that joint optimization of diverse tasks in multitask speech models has positive influence on the performance of individual tasks. In this paper we present a multitask speech model -- PolySpeech, which supports speech recognition, speech synthesis, and two speech classification tasks. PolySpeech takes multi-modal language model as its core structure and uses semantic representations as speech inputs. We introduce semantic speech embedding tokenization and speech reconstruction methods to PolySpeech, enabling efficient generation of high-quality speech for any given speaker. PolySpeech shows competitiveness across various tasks compared to single-task models. In our experiments, multitask optimization achieves performance comparable to single-task optimization and is especially beneficial for specific tasks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Mlphon: A Multifunctional Grapheme-Phoneme Conversion Tool Using Finite State Transducers
LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization
Thinking in Directivity: Speech Large Language Model for Multi-Talker Directional Speech Recognition
Seewo's Submission to MLC-SLM: Lessons learned from Speech Reasoning Language Models
NTU Speechlab LLM-Based Multilingual ASR System for Interspeech MLC-SLM Challenge 2025
Bi-directional Context-Enhanced Speech Large Language Models for Multilingual Conversational ASR
토론
이 논문과 연결된 포럼 글이 아직 없습니다.