Improve few-shot voice cloning using multi-modal learning
원문 초록
Recently, few-shot voice cloning has achieved a significant improvement. However, most models for few-shot voice cloning are single-modal, and multi-modal few-shot voice cloning has been understudied. In this paper, we propose to use multi-modal learning to improve the few-shot voice cloning performance. Inspired by the recent works on unsupervised speech representation, the proposed multi-modal system is built by extending Tacotron2 with an unsupervised speech representation module. We evaluate our proposed system in two few-shot voice cloning scenarios, namely few-shot text-to-speech(TTS) and voice conversion(VC). Experimental results demonstrate that the proposed multi-modal learning can significantly improve the few-shot voice cloning performance over their counterpart single-modal systems.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Few-Shot Speech Deepfake Detection Adaptation with Gaussian Processes
ClonEval: An Open Voice Cloning Benchmark
Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens
IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System
Metis: A Foundation Speech Generation Model with Masked Generative Pre-training
Overview of the Amphion Toolkit (v0.2)
토론
이 논문과 연결된 포럼 글이 아직 없습니다.