TIPAA-SSL: Text Independent Phone-to-Audio Alignment based on Self-Supervised Learning and Knowledge Transfer
원문 초록
In this paper, we present a novel approach for text independent phone-to-audio alignment based on phoneme recognition, representation learning and knowledge transfer. Our method leverages a self-supervised model (wav2vec2) fine-tuned for phoneme recognition using a Connectionist Temporal Classification (CTC) loss, a dimension reduction model and a frame-level phoneme classifier trained thanks to forced-alignment labels (using Montreal Forced Aligner) to produce multi-lingual phonetic representations, thus requiring minimal additional training. We evaluate our model using synthetic native data from the TIMIT dataset and the SCRIBE dataset for American and British English, respectively. Our proposed model outperforms the state-of-the-art (charsiu) in statistical metrics and has applications in language learning and speech processing systems. We leave experiments on other languages for future work but the design of the system makes it easily adaptable to other languages.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Fine-Tuning Self-Supervised Learning Models for End-to-End Pronunciation Scoring
These are Not All the Features You are Looking For: A Fundamental Bottleneck In Supervised Pretraining
Self-supervised Learning of Echocardiographic Video Representations via Online Cluster Distillation
ShapeEmbed: a self-supervised learning framework for 2D contour quantification
USAD: Universal Speech and Audio Representation via Distillation
Enhancing Few-shot Keyword Spotting Performance through Pre-Trained Self-supervised Speech Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.