Self-Supervised Representations for Singing Voice Conversion
원문 초록
A singing voice conversion model converts a song in the voice of an arbitrary source singer to the voice of a target singer. Recently, methods that leverage self-supervised audio representations such as HuBERT and Wav2Vec 2.0 have helped further the state-of-the-art. Though these methods produce more natural and melodic singing outputs, they often rely on confusion and disentanglement losses to render the self-supervised representations speaker and pitch-invariant. In this paper, we circumvent disentanglement training and propose a new model that leverages ASR fine-tuned self-supervised representations as inputs to a HiFi-GAN neural vocoder for singing voice conversion. We experiment with different f0 encoding schemes and show that an f0 harmonic generation module that uses a parallel bank of transposed convolutions (PBTC) alongside ASR fine-tuned Wav2Vec 2.0 features results in the best singing voice conversion quality. Additionally, the model is capable of making a spoken voice sing. We also show that a simple f0 shifting scheme during inference helps retain singer identity and bolsters the performance of our singing voice conversion model. Our results are backed up by extensive MOS studies that compare different ablations and baselines.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversion
REWIND: Speech Time Reversal for Enhancing Speaker Representations in Diffusion-based Voice Conversion
Eta-WavLM: Efficient Speaker Identity Removal in Self-Supervised Speech Representations Using a Simple Linear Equation
Adversarial Disentanglement by Backpropagation with Physics-Informed Variational Autoencoder
Dual-View Disentangled Multi-Intent Learning for Enhanced Collaborative Filtering
Training-Free Voice Conversion with Factorized Optimal Transport
토론
이 논문과 연결된 포럼 글이 아직 없습니다.