CrossVoice: Crosslingual Prosody Preserving Cascade-S2ST using Transfer Learning
원문 초록
This paper presents CrossVoice, a novel cascade-based Speech-to-Speech Translation (S2ST) system employing advanced ASR, MT, and TTS technologies with cross-lingual prosody preservation through transfer learning. We conducted comprehensive experiments comparing CrossVoice with direct-S2ST systems, showing improved BLEU scores on tasks such as Fisher Es-En, VoxPopuli Fr-En and prosody preservation on benchmark datasets CVSS-T and IndicTTS. With an average mean opinion score of 3.75 out of 4, speech synthesized by CrossVoice closely rivals human speech on the benchmark, highlighting the efficacy of cascade-based systems and transfer learning in multilingual S2ST with prosody transfer.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Scheduled Interleaved Speech-Text Training for Speech-to-Speech Translation with LLMs
S2ST-Omni: An Efficient and Scalable Multilingual Speech-to-Speech Translation Framework via Seamless Speech-Text Alignment and Streaming Speech Generation
Improving Language and Modality Transfer in Translation by Character-level Modeling
Dub-S2ST: Textless Speech-to-Speech Translation for Seamless Dubbing
TASK AWARE MULTI-TASK LEARNING FOR SPEECH TO TEXT TASKS
RaMen: Multi-Strategy Multi-Modal Learning for Bundle Construction
토론
이 논문과 연결된 포럼 글이 아직 없습니다.