Conditional variational autoencoder to improve neural audio synthesis for polyphonic music sound
원문 초록
Deep generative models for audio synthesis have recently been significantly improved. However, the task of modeling raw-waveforms remains a difficult problem, especially for audio waveforms and music signals. Recently, the realtime audio variational autoencoder (RAVE) method was developed for high-quality audio waveform synthesis. The RAVE method is based on the variational autoencoder and utilizes the two-stage training strategy. Unfortunately, the RAVE model is limited in reproducing wide-pitch polyphonic music sound. Therefore, to enhance the reconstruction performance, we adopt the pitch activation data as an auxiliary information to the RAVE model. To handle the auxiliary information, we propose an enhanced RAVE model with a conditional variational autoencoder structure and an additional fully-connected layer. To evaluate the proposed structure, we conducted a listening experiment based on multiple stimulus tests with hidden references and an anchor (MUSHRA) with the MAESTRO. The obtained results indicate that the proposed model exhibits a more significant performance and stability improvement than the conventional RAVE model.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Fast Differentiable Modal Simulation of Non-linear Strings, Membranes, and Plates
Generative diffusion model with inverse renormalization group flows
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis
OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows
Where are we in audio deepfake detection? A systematic analysis over generative and detection models
ViolinDiff: Enhancing Expressive Violin Synthesis with Pitch Bend Conditioning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.