Comparing Discrete and Continuous Space LLMs for Speech Recognition
원문 초록
This paper investigates discrete and continuous speech representations in Large Language Model (LLM)-based Automatic Speech Recognition (ASR), organizing them by feature continuity and training approach into four categories: supervised and unsupervised for both discrete and continuous types. We further classify LLMs based on their input and autoregressive feedback into continuous and discrete-space models. Using specialized encoders and comparative analysis with a Joint-Training-From-Scratch Language Model (JTFS LM) and pre-trained LLaMA2-7b, we provide a detailed examination of their effectiveness. Our work marks the first extensive comparison of speech representations in LLM-based ASR and explores various modeling techniques. We present an open-sourced achievement of a state-of-the-art Word Error Rate (WER) of 1.69\% on LibriSpeech using a HuBERT encoder, offering valuable insights for advancing ASR and natural language processing (NLP) research.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Bi-directional Context-Enhanced Speech Large Language Models for Multilingual Conversational ASR
Improving Named Entity Transcription with Contextual LLM-based Revision
Customizing Speech Recognition Model with Large Language Model Feedback
LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization
Seewo's Submission to MLC-SLM: Lessons learned from Speech Reasoning Language Models
Mlphon: A Multifunctional Grapheme-Phoneme Conversion Tool Using Finite State Transducers
토론
이 논문과 연결된 포럼 글이 아직 없습니다.