Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation
원문 초록
Diffusion models have achieved remarkable success in text-to-speech (TTS), even in zero-shot scenarios. Recent efforts aim to address the trade-off between inference speed and sound quality, often considered the primary drawback of diffusion models. However, we find a critical mispronunciation issue is being overlooked. Our preliminary study reveals the unstable pronunciation resulting from the diffusion process. Based on this observation, we introduce StableForm-TTS, a novel zero-shot speech synthesis framework designed to produce robust pronunciation while maintaining the advantages of diffusion modeling. By pioneering the adoption of source-filter theory in diffusion TTS, we propose an elaborate architecture for stable formant generation. Experimental results on unseen speakers show that our model outperforms the state-of-the-art method in terms of pronunciation accuracy and naturalness, with comparable speaker similarity. Moreover, our model demonstrates effective scalability as both data and model sizes increase. Audio samples are available online: https://deepbrainai-research.github.io/stableformtts/.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
RapFlow-TTS: Rapid and High-Fidelity Text-to-Speech with Improved Consistency Flow Matching
InstructTTSEval: Benchmarking Complex Natural-Language Instruction Following in Text-to-Speech Systems
ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching
Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models
MoonCast: High-Quality Zero-Shot Podcast Generation
Developing multilingual speech synthesis system for Ojibwe, Mi'kmaq, and Maliseet
토론
이 논문과 연결된 포럼 글이 아직 없습니다.