DiTSE: High-Fidelity Generative Speech Enhancement via Latent Diffusion Transformers
원문 초록
Real-world speech recordings suffer from degradations such as background noise and reverberation. Speech enhancement aims to mitigate these issues by generating clean high-fidelity signals. While recent generative approaches for speech enhancement have shown promising results, they still face two major challenges: (1) content hallucination, where plausible phonemes generated differ from the original utterance; and (2) inconsistency, failing to preserve speaker's identity and paralinguistic features from the input speech. In this work, we introduce DiTSE (Diffusion Transformer for Speech Enhancement), which addresses quality issues of degraded speech in full bandwidth. Our approach employs a latent diffusion transformer model together with robust conditioning features, effectively addressing these challenges while remaining computationally efficient. Experimental results from both subjective and objective evaluations demonstrate that DiTSE achieves state-of-the-art audio quality that, for the first time, matches real studio-quality audio from the DAPS dataset. Furthermore, DiTSE significantly improves the preservation of speaker identity and content fidelity, reducing hallucinations across datasets compared to state-of-the-art enhancers. Audio samples are available at: http://hguimaraes.me/DiTSE
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Mitigating Object Hallucinations via Sentence-Level Early Intervention
Robust One-step Speech Enhancement via Consistency Distillation
UQLM: A Python Package for Uncertainty Quantification in Large Language Models
MambAttention: Mamba with Multi-Head Attention for Generalizable Single-Channel Speech Enhancement
Mitigating Hallucination of Large Vision-Language Models via Dynamic Logits Calibration
KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality
토론
이 논문과 연결된 포럼 글이 아직 없습니다.