PyTorchKR Papers

최신 논문

총 124편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:1802.08435

Efficient Neural Audio Synthesis

Sequential models achieve state-of-the-art results in audio, visual and textual domains with respect to both estimating the data distribution and generating high-quality samples. Efficient sampling for this class of models has however remained an elusive problem. With a focus on text-to-speech synthesis, we describe a set of general techniques for reducing sampling time while maintaining high output quality. We first describe a single-layer recurrent neural network, the WaveRNN, with a dual softmax layer that matches the quality of the state-of-the-art WaveNet model. The compact form of the network makes it possible to generate 24kHz 16-bit audio 4x faster than real time on a GPU. Second, we apply a weight pruning technique to reduce the number of weights in the WaveRNN. We find that, for a constant number of parameters, large sparse networks perform better than small dense networks and this relationship holds for sparsity levels beyond 96%. The small number of weights in a Sparse WaveRNN makes it possible to sample high-fidelity audio on a mobile CPU in real time. Finally, we propose a new generation scheme based on subscaling that folds a long sequence into a batch of shorter sequences and allows one to generate multiple samples at once. The Subscale WaveRNN produces 16 samples per step without loss of quality and offers an orthogonal method for increasing sampling efficiency.

arXiv:1803.01271공식 코드 · Papers with Code

An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

For most deep learning practitioners, sequence modeling is synonymous with recurrent networks. Yet recent results indicate that convolutional architectures can outperform recurrent networks on tasks such as audio synthesis and machine translation. Given a new sequence modeling task or dataset, which architecture should one use? We conduct a systematic evaluation of generic convolutional and recurrent architectures for sequence modeling. The models are evaluated across a broad range of standard tasks that are commonly used to benchmark recurrent networks. Our results indicate that a simple convolutional architecture outperforms canonical recurrent networks such as LSTMs across a diverse range of tasks and datasets, while demonstrating longer effective memory. We conclude that the common association between sequence modeling and recurrent networks should be reconsidered, and convolutional networks should be regarded as a natural starting point for sequence modeling tasks. To assist related work, we have made code available at http://github.com/locuslab/TCN .

arXiv:1805.10808

Real-valued parametric conditioning of an RNN for interactive sound synthesis

A Recurrent Neural Network (RNN) for audio synthesis is trained by augmenting the audio input with information about signal characteristics such as pitch, amplitude, and instrument. The result after training is an audio synthesizer that is played like a musical instrument with the desired musical characteristics provided as continuous parametric control. The focus of this paper is on conditioning data-driven synthesis models with real-valued parameters, and in particular, on the ability of the system a) to generalize and b) to be responsive to parameter values and sequences not seen during training.

arXiv:2506.20995v4cs.CV

부정적 오디오 가이던스를 통한 단계별 비디오-오디오 합성

Step-by-Step Video-to-Audio Synthesis via Negative Audio Guidance

우리는 생성 과정에 대한 더욱 세밀한 제어와 더욱 현실적인 오디오 합성을 제공하는 단계별 비디오-오디오(V2A) 생성 방법을 제안한다. 전통적인 폴리(Foley) 워크플로에서 영감을 받은 본 접근법은 보완적인 사운드를 점진적으로 생성할 수 있게 하여, 사용자가 비디오에 의해 유발된 여러 사운드 이벤트를 저작할 수 있도록 한다. 비용이 많이 드는 다중 참조 비디오-오디오 데이터셋의 필요성을 피하기 위해, 각 생성 단계는 이전에 생성된 트랙에 이미 존재하는 사운드의 중복을 억제하는 부정적 가이던스 V2A 프로세스로 공식화된다. 가이던스 모델은 동일 비디오의 비중첩 구간에서 추출한 오디오 쌍을 이용해 사전학습된 V2A 모델을 파인튜닝하여 학습되며, 시각적 그라운딩을 유지하면서 음향적 컨텍스트를 활용하도록 장려하고, 표준 단일 참조 시청각 데이터셋으로의 학습을 가능하게 한다. 객관적 및 주관적 평가를 통해 본 방법이 각 단계에서 생성된 사운드의 분리 가능성을 향상시키고 최종 복합 오디오의 전체적인 품질을 개선하며 기존 기준선을 능가함을 입증한다. 프로젝트 페이지: https://ahykw.github.io/sbsv2a/