TTS-Guided Training for Accent Conversion Without Parallel Data
원문 초록
Accent Conversion (AC) seeks to change the accent of speech from one (source) to another (target) while preserving the speech content and speaker identity. However, many AC approaches rely on source-target parallel speech data. We propose a novel accent conversion framework without the need of parallel data. Specifically, a text-to-speech (TTS) system is first pretrained with target-accented speech data. This TTS model and its hidden representations are expected to be associated only with the target accent. Then, a speech encoder is trained to convert the accent of the speech under the supervision of the pretrained TTS model. In doing so, the source-accented speech and its corresponding transcription are forwarded to the speech encoder and the pretrained TTS, respectively. The output of the speech encoder is optimized to be the same as the text embedding in the TTS system. At run-time, the speech encoder is combined with the pretrained TTS decoder to convert the source-accented speech toward the target. In the experiments, we converted English with two source accents (Chinese and Indian) to the target accent (American/British/Canadian). Both objective metrics and subjective listening tests successfully validate that, without any parallel data, the proposed approach generates speech samples that are close to the target accent with high speech quality.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching
SpeakStream: Streaming Text-to-Speech with Interleaved Data
Effective Deep Learning Models for Automatic Diacritization of Arabic Text
ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching
Differentiable Reward Optimization for LLM based TTS system
PresentAgent: Multimodal Agent for Presentation Video Generation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.