Papers with Code 아카이브

Speech Recognition

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크
11
평가표
236
보존된 평가 기록
398

Speech Recognition is the task of converting spoken language into text. It involves recognizing the words spoken in an audio recording and transcribing them into a written format. The goal is to accurately transcribe the speech in real-time or from recorded audio, taking into account factors such as accents, speaking speed, and background noise.

<span style="color:grey; opacity: 0.6">( Image credit: SpecAugment )</span>

데이터셋

출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.

RealMAN

The Audio Signal and Information Processing Lab at Westlake University, in collaboration with AISHELL, has released the Real-recorded and annotated Microphone Array speech&Noise (RealMAN) dataset, which provides annotated multi-channel speech and noise recordings for dynamic speech enhancement and localization: - Microphone array: A 32-channel microphone array with high-fidelity microphones is used for recording - Speech source: A loudspeaker is used for playing source speech signals (about 35 hours of Mandarin speech) - Recording duration and scene: A total of 83.7 hours of speech signals (about 48.3 hours for static speaker and 35.4 hours for moving speaker) are recorded in 32 different scenes, and 144.5 hours of background noise are recorded in 31 different scenes. Both speech and noise recording scenes cover various common indoor, outdoor, semi-outdoor and transportation environments, which enables the training of general-purpose speech enhancement and source localization networks. - Annotation: To obtain the task-specific annotations, speaker location is annotated with an omni-directional fisheye camera by automatically detecting the loudspeaker. The direct-path signal is set as the target clean speech for speech enhancement, which is obtained by filtering the source speech signal with an estimated direct-path propagation filter.

Timers and Such

Timers and Such is an open source dataset of spoken English commands for common voice control use cases involving numbers. The dataset has four intents, corresponding to four common offline voice assistant uses: SetTimer, SetAlarm, SimpleMath, and UnitConversion. The semantic label for each utterance is a dictionary with the intent and a number of slots. All recordings were converted from their original formats to single-channel 16,000 Hz .wav files.

word2word

word2word contains easy-to-use word translations for 3,564 language pairs. - A large collection of freely & publicly available bilingual lexicons for 3,564 language pairs across 62 unique languages. - Easy-to-use Python interface for accessing top-k word translations and for building a new bilingual lexicon from a custom parallel corpus. - Constructed using a simple approach that yields bilingual lexicons with high coverage and competitive translation quality. Source: word2word

EdAcc

The Edinburgh International Accents of English Corpus (EdAcc) is a new automatic speech recognition (ASR) dataset composed of 40 hours of English dyadic conversations between speakers with a diverse set of accents. EdAcc includes a wide range of first and second-language varieties of English and a linguistic background profile of each speaker. Source: The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR Image Source: The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR

ITALIC

ITALIC: An ITALian Intent Classification Dataset ITALIC is an intent classification dataset for the Italian language, which is the first of its kind. It includes spoken and written utterances and is annotated with 60 intents. The dataset is available on Zenodo and connectors ara available for the HuggingFace Hub. ### Data collection The data collection follows the MASSIVE NLU dataset which contains an annotated textual dataset for 60 intents. The data collection process is described in the paper Massive Natural Language Understanding. Following the MASSIVE NLU dataset, a pool of 70+ volunteers has been recruited to annotate the dataset. The volunteers were asked to record their voice while reading the utterances (the original text is available on MASSIVE dataset). Together with the audio, the volunteers were asked to provide a self-annotated description of the recording conditions (e.g., background noise, recording device). The audio recordings have also been validated and, in case of errors, re-recorded by the volunteers. All the audio recordings included in the dataset have received a validation from at least two volunteers. All the audio recordings have been validated by native italian speakers (self-annotated).

RTASC

The ROBIN Technical Acquisition Speech Corpus (ROBINTASC) was developed within the ROBIN project. Its main purpose was to improve the behaviour of a conversational agent, allowing human-machine interaction in the context of purchasing technical equipment. It contains over 6 hours of read speech in Romanian language. We provide text files, associated speech files (WAV, 44.1KHz, 16-bit, single channel), annotated text files in CoNLL-U format.

VietMed

We introduced a Vietnamese speech recognition dataset in the medical domain comprising 16h of labeled medical speech, 1000h of unlabeled medical speech and 1200h of unlabeled general-domain speech. To our best knowledge, VietMed is by far the world’s largest public medical speech recognition dataset in 7 aspects: total duration, number of speakers, diseases, recording conditions, speaker roles, unique medical terms and accents. VietMed is also by far the largest public Vietnamese speech dataset in terms of total duration. Additionally, we are the first to present a medical ASR dataset covering all ICD-10 disease groups and all accents within a country.

ArzEn

Corpus of Egyptian Arabic-English Code-switching (ArzEn) is a spontaneous conversational speech corpus, obtained through informal interviews held at the German University in Cairo. The participants discussed broad topics, including education, hobbies, work, and life experiences. The corpus currently contains 12 hours of speech, having 6,216 utterances. The recordings were transcribed and translated into monolingual Egyptian Arabic and monolingual English. Source: https://arxiv.org/pdf/2211.16319v1.pdf

EMOVIE

EMOVIE is a Mandarin emotion speech dataset including 9,724 samples with audio files and its emotion human-labeled annotation.

Google Speech Commands - Musan

This noisy speech test set is created from the Google Speech Commands v2 [1] and the Musan dataset[2]. It could be downloaded here: https://zenodo.org/record/6066174#.Yn7NPJPMLyU Specifically, we created this test set by mixing the speech in the Google Speech Commands v2 test set with random noise in the Musan dataset at different signal to noise ratio -12.5,-10,0,10,20,30 and 40 decibel (dB). The Google Speech Commands v2 dataset is under the Creative Commons BY 4.0 license. It could be downloaded at: http://download.tensorflow.org/data/speech_commands_v0.02.tar.gz The Musan dataset is under Attribution 4.0 International (CC BY 4.0). It could be downlowned at https://www.openslr.org/17/ Citations: [1] Pete Warden, “Speech commands: A dataset for limited-vocabulary speech recognition,” arXiv preprint arXiv:1804.03209, 2018. [2] David Snyder, Guoguo Chen, and Daniel Povey, “Musan: A music, speech, and noise corpus,” arXiv preprint arXiv:1510.08484, 2015.

HUI speech corpus

The data set contains several speakers. The 5 largest are listed individually, the rest are summarized as other. All audio files have a sampling rate of 44.1kHz. For each speaker, there is a clean variant in addition to the full data set, where the quality is even higher. Furthermore, there are various statistics. The dataset can also be used for automatic speech recognition (ASR) if audio files are converted to 16 kHz.

IndicTTS

A special corpus of Indian languages covering 13 major languages of India. It comprises of 10000+ spoken sentences/utterances each of mono and English recorded by both Male and Female native speakers. Speech waveform files are available in .wav format along with the corresponding text. We hope that these recordings will be useful for researchers and speech technologists working on synthesis and recognition. You can request zip archives of the entire database here.

LibriVoxDeEn

LibriVoxDeEn is a corpus of sentence-aligned triples of German audio, German text, and English translation, based on German audiobooks. The speech translation data consist of 110 hours of audio material aligned to over 50k parallel sentences. An even larger dataset comprising 547 hours of German speech aligned to German text is available for speech recognition. The audio data is read speech and thus low in disfluencies. Source: LibriVoxDeEn: A Corpus for German-to-English Speech Translation and German Speech Recognition

NusaCrowd

NusaCrowd is a collaborative initiative to collect and unite existing resources for Indonesian languages, including opening access to previously non-public resources. Through this initiative, the authors have has brought together 137 datasets and 117 standardized data loaders. The quality of the datasets has been assessed manually and automatically, and their effectiveness has been demonstrated in multiple experiments. Source: https://arxiv.org/pdf/2212.09648v2.pdf Image Source: NusaCrowd: Open Source Initiative for Indonesian NLP Resources

RESD

Russian dataset of emotional speech dialogues. This dataset was assembled from ~3.5 hours of live speech by actors who voiced pre-distributed emotions in the dialogue for ~3 minutes each. <br> Each sample of dataset contains name of part from the original dataset studio source, speech file (16000 or 44100Hz) of human voice, 1 of 7 labeled emotions and the speech-to-texted part of voice speech. <br> Emotions are represented in 7 states: anger, disgust, fear, enthusiasm, happiness, neutral and sadness. This dataset was created by Artem Amentes, Nikita Davidchuk and Ilya Lubenets @misc{Aniemore, author = {Артем Аментес, Илья Лубенец, Никита Давидчук}, title = {Открытая библиотека искусственного интеллекта для анализа и выявления эмоциональных оттенков речи человека}, year = {2022}, publisher = {Hugging Face}, journal = {Hugging Face Hub}, howpublished = {\url{https://huggingface.com/aniemore/Aniemore}}, email = {hello@socialcode.ru} }

TaL Corpus

The Tongue and Lips (TaL) corpus is a multi-speaker corpus of ultrasound images of the tongue and video images of lips. This corpus contains synchronised imaging data of extraoral (lips) and intraoral (tongue) articulators from 82 native speakers of English. The TaL corpus consists of two datasets: * TaL1 is a single-speaker dataset containing data of one professional voice talent, a male native speaker of English, over six recording sessions. * TaL80 is a multi-speaker dataset contains recording sessions of 81 native speakers of English without voice talent experience. Each speaker was recording over a single recording session. Image source: https://ultrasuite.github.io/data/tal_corpus/

Zambezi Voice

This work introduces Zambezi Voice, an open-source multilingual speech resource for Zambian languages. It contains two collections of datasets: unlabelled audio recordings of radio news and talk shows programs (160 hours) and labelled data (over 80 hours) consisting of read speech recorded from text sourced from publicly available literature books. The dataset is created for speech recognition but can be extended to multilingual speech processing research for both supervised and unsupervised learning approaches. To our knowledge, this is the first multilingual speech dataset created for Zambian languages. We exploit pretraining and cross-lingual transfer learning by finetuning the Wav2Vec2.0 large-scale multilingual pre-trained model to build end-to-end (E2E) speech recognition models for our baseline models. The dataset is released publicly under a Creative Commons BY-NC-ND 4.0 license and can be accessed through the project repository.

AV Digits Database

AV Digits Database is an audiovisual database which contains normal, whispered and silent speech. 53 participants were recorded from 3 different views (frontal, 45 and profile) pronouncing digits and phrases in three speech modes. The database consists of two parts: digits and short phrases. In the first part, participants were asked to read 10 digits, from 0 to 9, in English in random order five times. In case of non-native English speakers this part was also repeated in the participant’s native language. In total, 53 participants (41 males and 12 females) from 16 nationalities, were recorded with a mean age and standard deviation of 26.7 and 4.3 years, respectively. In the second part, participants were asked to read 10 short phrases. The phrases are the same as the ones used in the OuluVS2 database: “Excuse me”, “Goodbye”, “Hello”, “How are you”, “Nice to meet you”, “See you”, “I am sorry”, “Thank you”, “Have a good time”, “You are welcome”. Again, each phrase was repeated five times in 3 different modes, neutral, whisper and silent speech. Thirty nine participants (32 males and 7 females) were recorded for this part with a mean age and standard deviation of 26.3 and 3.8 years, respectively. Source: AV Digits Database

GOLOS

Golos is a Russian speech dataset suitable for speech research. The dataset mainly consists of recorded audio files manually annotated on the crowd-sourcing platform. The total duration of the audio is about 1240 hours. ## Dataset structure | Domain | Train files | Train hours | Test files | Test hours | |----------------|------------|--------|-------|------| | Crowd | 979 796 | 1 095 | 9 994 | 11.2 | | Farfield | 124 003 | 132.4| 1 916 | 1.4 | | Total | 1 103 799 | 1 227.4|11 910 | 12.6 | ### Audio files in opus format | Archive | Size | Link | |------------------|------------|---------------------| | golos_opus.tar | 20.5 GB | https://sc.link/JpD | ### Audio files in wav format | Archives | Size | Links | |-------------------|------------|---------------------| | train_farfield.tar| 15.4 GB | https://sc.link/1Z3 | | train_crowd0.tar | 11 GB | https://sc.link/Lrg | | train_crowd1.tar | 14 GB | https://sc.link/MvQ | | train_crowd2.tar | 13.2 GB | https://sc.link/NwL | | train_crowd3.tar | 11.6 GB | https://sc.link/Oxg | | train_crowd4.tar | 15.8 GB | https://sc.link/Pyz | | train_crowd5.tar | 13.1 GB | https://sc.link/Qz7 | | train_crowd6.tar | 15.7 GB | https://sc.link/RAL | | train_crowd7.tar | 12.7 GB | https://sc.link/VG5 | | train_crowd8.tar | 12.2 GB | https://sc.link/WJW | | train_crowd9.tar | 8.08 GB | https://sc.link/XKk | | test.tar | 1.3 GB | https://sc.link/Kqr | ## Evaluation Percents of Word Error Rate for different test sets | Decoder \ Test set | Crowd test | Farfield test | MCV<sup>1</sup> dev | MCV<sup>1</sup> test | |-------------------------------------|-----------|----------|-----------|----------| | Greedy decoder | 4.389 % | 14.949 % | 9.314 % | 11.278 % | | Beam Search with Common Crawl LM | 4.709 % | 12.503 % | 6.341 % | 7.976 % | | Beam Search with Golos train set LM | 3.548 % | 12.384 % | - | - | | Beam Search with Common Crawl and Golos LM | 3.318 % | 11.488 % | 6.4 % | 8.06 % |

Jam-ALT

JamALT is a revision of the JamendoLyrics dataset (80 songs in 4 languages), adapted for use as an automatic lyrics transcription (ALT) benchmark. The lyrics have been revised according to the newly compiled annotation guide, which include rules about spelling, punctuation, and formatting. The audio is identical to the JamendoLyrics dataset. However, only 79 songs are included, as one of the 20 French songs has been removed due to concerns about potentially harmful content.

JVS-MuSiC

JVS-MuSiC is a Japanese multispeaker singing-voice corpus called "JVS-MuSiC" with the aim to analyze and synthesize a variety of voices. The corpus consists of 100 singers' recordings of the same song, Katatsumuri, which is a Japanese children's song. It also includes another song that is different for each singer.

MASRI-HEADSET

MASRI-HEADSET is a corpus that was developed by the MASRI project at the University of Malta. It consists of 8 hours of speech paired with text, recorded by using short text snippets in a laboratory environment. The speakers were recruited from different geographical locations all over the Maltese islands, and were roughly evenly distributed by gender. Source: MASRI-HEADSET: A Maltese Corpus for Speech Recognition

NeuroVoz

The NeuroVoz dataset emerges as a pioneering resource in the field of computational linguistics and biomedical research, specifically designed to enhance the diagnosis and understanding of Parkinson's Disease (PD) through speech analysis. This dataset is distinguished as the first of its kind to be made publicly available in Castilian Spanish, addressing a critical gap in the availability of linguistic and dialectical diversity within PD research. Compiled from a cohort of 112 participants, including 54 individuals diagnosed with PD and 58 healthy controls, the NeuroVoz dataset offers a rich compilation of speech recordings. All PD participants were recorded under medication (ON state), ensuring consistency and reliability in the speech samples collected. The dataset is meticulously curated to include a variety of speech tasks—ranging from sustained vowel phonations and diadochokinetic (DDK) tests to 16 structured listen-and-repeat utterances and spontaneous monologues. The inclusion of both manually transcribed listen-and-repeat tasks and Whisper-automated transcriptions for monologues underscores our commitment to data accuracy and usability. Encompassing 2,977 audio files, the NeuroVoz dataset provides an extensive repository, averaging 26.88 +- 3.35 recordings per participant, making it an invaluable asset for researchers seeking to explore the nuances of PD-affected speech. The dataset's structure and composition facilitate a multifaceted analysis of speech impairments associated with PD, offering insights into phonatory, articulatory, and prosodic changes. In contributing to the body of knowledge with the NeuroVoz dataset, we invite the scientific community to engage with this dataset, explore the specific speech characteristics of PD in Castilian Spanish speakers, and advance the field of PD diagnosis through innovative speech analysis techniques. Paper explaining the database: https://arxiv.org/abs/2403.02371 \ Zenodo repository with the database: https://zenodo.org/records/13647600 \ Github repo on how to use the dataset: https://github.com/BYO-UPM/Neurovoz_Dababase

NPSC

The Norwegian Parliamentary Speech Corpus (NPSC) is a speech corpus made by the Norwegian Language Bank at the National Library of Norway in 2019-2021. The NPSC consists of recordings of speech from Stortinget, the Norwegian parliament, and corresponding orthographic transcriptions to Norwegian Bokmål and Norwegian Nynorsk. All transcriptions are done manually by trained linguists or philologists, and the manual transcriptions are subsequently proofread to ensure consistency and accuracy. Entire days of Parliamentary meetings are transcribed in the dataset.

OpenSLR

OpenSLR is a repository of open speech and language resources, including large-scale transcribed audio corpora and related software. It serves as a central platform for researchers and practitioners to access and share datasets used in speech recognition (ASR), text-to-speech (TTS), and linguistic research. The OpenSLR collection includes over 30 diverse datasets spanning more than 25 languages, such as Javanese, Nepali, Malayalam, Yoruba, and various English and Spanish dialects. These datasets are contributed by institutions including Google, North West University, and others. Most contain audio recordings along with transcriptions, covering both crowd-sourced and professionally recorded material. Many of the datasets are high-quality multi-speaker corpora intended for use in building ASR and TTS models, particularly for under-resourced languages. Use cases include multilingual speech recognition, dialect modeling, language technology research, and building open-source voice applications. The OpenSLR site also acts as a mirror for widely-used tools and models to ensure continued availability. Website: OpenSLR website Source: openslr/openslr

RUSLAN

RUSLAN is a Russian spoken language corpus for text-to-speech task. RUSLAN contains 22,200 audio samples with text annotations – more than 31 hours of high-quality speech of one person – being one of the largest annotated Russian corpus in terms of speech duration for a single speaker. Source: RUSLAN: Russian Spoken Language Corpus for Speech Synthesis

Tilde MODEL Corpus

Tilde MODEL Corpus is a multilingual corpora for European languages – particularly focused on the smaller languages. The collected resources have been cleaned, aligned, and formatted into a corpora standard TMX format useable for developing new Language technology products and services. It contains over 10M segments of multilingual open data. The data has been collected from sites allowing free use and reuse of its content, as well as from Public Sector web sites. Source: Tilde MODEL - Multilingual Open Data for EU Languages

TTS-Portuguese Corpus

The dataset has 10.5 hours from a single speaker. Source: End-To-End Speech Synthesis Applied to Brazilian Portuguese

VocBench

VocBench is a framework that benchmark the performance of state-of-the art neural vocoders. VocBench uses a systematic study to evaluate different neural vocoders in a shared environment that enables a fair comparison between them.

Arabic Speech Commands Dataset

This dataset is designed to help train simple machine learning models that serve educational and research purposes in the speech recognition domain, mainly for keyword spotting tasks. ## Dataset Description Our dataset is a list of pairs (x, y), where x is the input speech signal, and y is the corresponding keyword. The final dataset consists of 12000 such pairs, comprising 40 keywords. Each audio file is one-second in length sampled at 16 kHz. We have 30 participants, each of them recorded 10 utterances for each keyword. Therefore, we have 300 audio files for each keyword in total (30 * 10 * 40 = 12000), and the total size of all the recorded keywords is ~384 MB. The dataset also contains several background noise recordings we obtained from various natural sources of noise. We saved these audio files in a separate folder with the name background_noise and a total size of ~49 MB. ## Dataset Structure There are 40 folders, each of which represents one keyword and contains 300 files. The first eight digits of each file name identify the contributor, while the last two digits identify the round number. For example, the file path rotate/00000021_NO_06.wav indicates that the contributor with the ID 00000021 pronounced the keyword rotate for the 6th time. ## Data Split We recommend using the provided CSV files in your experiments. We kept 60% of the dataset for training, 20% for validation, and the remaining 20% for testing. In our split method, we guarantee that all recordings of a certain contributor are within the same subset. ## License This dataset is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. For more details, see the LICENSE file in this folder. ## Citations If you want to use the Arabic Speech Commands dataset in your work, please cite it as: @article{ghandoura2021building, title={Building and benchmarking an Arabic Speech Commands dataset for small-footprint keyword spotting}, author={Ghandoura, Abdulkader and Hjabo, Farouk and Al Dakkak, Oumayma}, journal={Engineering Applications of Artificial Intelligence}, volume={102}, pages={104267}, year={2021}, publisher={Elsevier} }

BERSt

BERSt Dataset We release the BERSt Dataset for various speech recognition tasks including Automatic Speech Recognition (ASR) and Speech Emotion Recogniton (SER) Overview 4526 single phrase recordings (~3.75h) 98 professional actors 19 phone positions 7 emotion classes 3 vocal intensity levels varied regional and non-native English accents nonsense phrases covering all English Phonemes Data collection The BERSt dataset represents data collected in home envrionments using various smartphone microphones (phone model available as metadata) Participants were around the globe and represent varying regional accents in English: UK, Canada, USA (multi-state), Australia, including a subset of the data that is non-native English speakers including: French, Russian, Hindi etc. The data includes 13 non-sense phrases for use cases robust to lingustic context and high surprisal. Partipants were prompted to speak, raise their voice and shout each phrase while moving their phone to various distances and locations in their home, as well as with various obstructions to the microphone, e.g. in a backpack Baseline results of various state-of-the-art methods for ASR and SER show that this dataset remains a challenging task, and we encourage researchers to use this data to fine-tune and benchmark their models in these difficult condition representing possible real world situations Affect annotations are those provided to the actors, they have not been validated through perception The speech annotations, however, has been checked and adjusted to mistakes in the speech. Data splits and organisation For each phone position and phrase, the actors provided a single recording for the three vocal intensity levels, these raw audio files are available Meta-data in csv format corresponds to the files split per utterance with noise and silence before and after speech removed, found inside clean_clips for each data splits We provide a test, train and validation split There is no speaker cross-over between splits, the train and validation sets each contain 10 speakers not seen in the training set Metadata Details actor count 98 Gender counts Woman: 61 Man: 34 Non-Binary: 1 Prefer not to disclose 2 Current daily language counts English: 95 Norwegian: 1 Russian: 1 French: 1 First language counts English: 75 Non English: 23 Spanish: 6 French: 3 Portuguese: 3 Chinese: 2 Norwegian: 1 Mandarin: 1 Tagalog: 1 Italian: 1 Hungarian: 1 Russian: 1 Hindi: 1 Swahili: 1 Croatian: 1 Pre-split Data counts Emotion counts fear: 236 neutral: 234 disgust: 232 joy: 224 anger: 223 surprise: 210 sadness: 201 Distance counts: Near body: 627 1-2m away: 324 Other side of room: 316 Outside of room: 293

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.