vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoders
원문 초록
We propose a new speech discrete token vocoder, vec2wav 2.0, which advances voice conversion (VC). We use discrete tokens from speech self-supervised models as the content features of source speech, and treat VC as a prompted vocoding task. To amend the loss of speaker timbre in the content tokens, vec2wav 2.0 utilizes the WavLM features to provide strong timbre-dependent information. A novel adaptive Snake activation function is proposed to better incorporate timbre into the waveform reconstruction process. In this way, vec2wav 2.0 learns to alter the speaker timbre appropriately given different reference prompts. Also, no supervised data is required for vec2wav 2.0 to be effectively trained. Experimental results demonstrate that vec2wav 2.0 outperforms all other baselines to a considerable margin in terms of audio quality and speaker similarity in any-to-any VC. Ablation studies verify the effects made by the proposed techniques. Moreover, vec2wav 2.0 achieves competitive cross-lingual VC even only trained on monolingual corpus. Thus, vec2wav 2.0 shows timbre can potentially be manipulated only by speech token vocoders, pushing the frontiers of VC and speech synthesis.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Region-Based Optimization in Continual Learning for Audio Deepfake Detection
SALF-MOS: Speaker Agnostic Latent Features Downsampled for MOS Prediction
ArVoice: A Multi-Speaker Dataset for Arabic Speech Synthesis
Investigating self-supervised features for expressive, multilingual voice conversion
Generative Adversarial Network based Voice Conversion: Techniques, Challenges, and Recent Advancements
FADEL: Uncertainty-aware Fake Audio Detection with Evidential Deep Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.