Speech Separation

VoxCeleb2 is a large scale speaker recognition dataset obtained automatically from open-source media. VoxCeleb2 consists of over a million utterances from over 6k speakers. Since the dataset is collected ‘in the wild’, the speech segments are corrupted with real world noise including laughter, cross-talk, channel effects, music and other sounds. The dataset is also multilingual, with speech from speakers of 145 different nationalities, covering a wide range of accents, ages, ethnicities and languages. The dataset is audio-visual, so is also useful for a number of other applications, for example – visual speech synthesis, speech separation, cross-modal transfer from face to voice or vice versa and training face recognition from video to complement existing face recognition datasets. Source: VoxCeleb2: Deep Speaker Recognition Image Source: https://www.robots.ox.ac.uk/~vgg/data/voxceleb/

WSJ0-2mix is a speech recognition corpus of speech mixtures using utterances from the Wall Street Journal (WSJ0) corpus. Source: Deep clustering: Discriminative embeddings for segmentation and separation

LibriMix is an open-source alternative to wsj0-2mix. Based on LibriSpeech, LibriMix consists of two- or three-speaker mixtures combined with ambient noise samples from WHAM!. Source: LibriMix: An Open-Source Dataset for Generalizable Speech Separation

The Oxford-BBC Lip Reading Sentences 2 (LRS2) dataset is one of the largest publicly available datasets for lip reading sentences in-the-wild. The database consists of mainly news and talk shows from BBC programs. Each sentence is up to 100 characters in length. The training, validation and test sets are divided according to broadcast date. It is a challenging set since it contains thousands of speakers without speaker labels and large variation in head pose. The pre-training set contains 96,318 utterances, the training set contains 45,839 utterances, the validation set contains 1,082 utterances and the test set contains 1,242 utterances. Source: Audio-visual Recognition of Overlapped speech for the LRS2 dataset Image Source: https://www.robots.ox.ac.uk/~vgg/data/lip_reading/lrs2.html

The WSJ0 Hipster Ambient Mixtures (WHAM!) dataset pairs each two-speaker mixture in the wsj0-2mix dataset with a unique noise background scene. It has an extension called WHAMR! that adds artificial reverberation to the speech signals in addition to the background noise. The noise audio was collected at various urban locations throughout the San Francisco Bay Area in late 2018. The environments primarily consist of restaurants, cafes, bars, and parks. Audio was recorded using an Apogee Sennheiser binaural microphone on a tripod between 1.0 and 1.5 meters off the ground.

Continuous speech separation (CSS) is an approach to handling overlapped speech in conversational audio signals. A real recorded dataset, called LibriCSS, is derived from LibriSpeech by concatenating the corpus utterances to simulate a conversation and capturing the audio replays with far-field microphones. Source: https://github.com/chenzhuo1011/libri_css

WHAMR! is a dataset for noisy and reverberant speech separation. It extends WHAM! by introducing synthetic reverberation to the speech sources in addition to the existing noise. Room impulse responses were generated and convolved using pyroomacoustics. Reverberation times were chosen to approximate domestic and classroom environments (expected to be similar to the restaurants and coffee shops where the WHAM! noise was collected), and further classified as high, medium, and low reverberation based on a qualitative assessment of the mixture’s noise recording.

The TIMIT Acoustic-Phonetic Continuous Speech Corpus is a standard dataset used for evaluation of automatic speech recognition systems. It consists of recordings of 630 speakers of 8 dialects of American English each reading 10 phonetically-rich sentences. It also comes with the word and phone-level transcriptions of the speech. Source: Improving neural networks by preventing co-adaptation of feature detectors Image Source: https://roboticrun.wordpress.com/2016/06/21/timit-introduction-the-official-doc/

MIR-1K (Multimedia Information Retrieval lab, 1000 song clips) is a dataset designed for singing voice separation. It contains: * 1000 song clips with the music accompaniment and the singing voice recorded as left and right channels, respectively, * Manual annotations of pitch contours in semitone, indices and types for unvoiced frames, lyrics, and vocal/non-vocal segments, * The speech recordings of the lyrics by the same person who sang the songs. The duration of each clip ranges from 4 to 13 seconds, and the total length of the dataset is 133 minutes. These clips are extracted from 110 karaoke songs which contain a mixture track and a music accompaniment track. These songs are freely selected from 5000 Chinese pop songs and sung by researchers from MIR lab (8 females and 11 males). Most of the singers are amateur and do not have professional music training. Source: https://sites.google.com/site/unvoicedsoundseparation/mir-1k Audio Source: https://sites.google.com/site/unvoicedsoundseparation/sounddemosforjournal

The iKala dataset is a singing voice separation dataset that comprises of 252 30-second excerpts sampled from 206 iKala songs (plus 100 hidden excerpts reserved for MIREX data mining contest). The music accompaniment and the singing voice are recorded at the left and right channels respectively. Additionally, the human-labeled pitch contours and timestamped lyrics are provided. This dataset is not available anymore. Source: http://mac.citi.sinica.edu.tw/ikala/ 1. "T" as a sofa: The "T" horizontal strip can mimic the back of a sofa with a delicate cushion or details of the uphols or appliances with the color button. The "T" vertical strip can show a feet or arm of the sofa, shiny, yet firm. 2. Merge "P": Put "P" next to "T", your curve to delicately with the top "T." It is intertwined. The circular part of "P" can show a cushion or a curved chair and synchronize the subject of furniture. Make sure "P" is visually relying on "T", which reflects the relationship of cohesion and balance. 5. Coherence of "B" and "I": "B" can be aligned as a pair of cushions or a modern chair, with mild curves with glossy and modern aesthetics. "I" can be a symbol of a shiny furniture or a vertical light bar and completes the shapes without overburdess them. Color palette 4: Includes soft soil colors such as beige, top and gray shades, along with silent or silver gold tips to touch elegance. Consider a slope effect to enhance modernity, to keep colors elegant and complex. 5. Connect the letters: Use the overlap or intertwined edges that the letters meet for the symbol of unity. The plan should allow viewers to distinguish each letter while feeling part of the same "structure". 6. Background patterns: Use delicate geometric patterns or textures that mimic fabrics or furniture materials such as wood seeds or woven fibers. These patterns must remain minimalist and focus on highlighting the logo, while maintaining communication. While it deals with the subject of furniture and design, this concept conveys modernity, creativity and professional. If you like, I can create a draft design for better visualization.

We present a speech data corpus that simulates a "dinner party" scenario taking place in an everyday home environment. The corpus was created by recording multiple groups of four Amazon employee volunteers having a natural conversation in English around a dining table. The participants were recorded by a single-channel close-talk microphone and by five far-field 7-microphone array devices positioned at different locations in the recording room. The dataset contains the audio recordings and human labeled transcripts of a total of 10 sessions with a duration between 15 and 45 minutes. The corpus was created to advance in the field of noise robust and distant speech processing and is intended to serve as a public research and benchmarking data set.

The QMUL underGround Re-IDentification (GRID) dataset contains 250 pedestrian image pairs. Each pair contains two images of the same individual seen from different camera views. All images are captured from 8 disjoint camera views installed in a busy underground station. The figures beside show a snapshot of each of the camera views of the station and sample images in the dataset. The dataset is challenging due to variations of pose, colours, lighting changes; as well as poor image quality caused by low spatial resolution. Source: https://personal.ie.cuhk.edu.hk/~ccloy/downloads_qmul_underground_reid.html

WSJ0-2mix-extr is a speech extraction dataset

Real-M is a crowd-sourced speech-separation corpus of real-life mixtures. The mixtures are recorded in different acoustic environments using a wide variety of recording devices such as laptops and smartphones, thus reflecting more closely potential application scenarios.

Here we release the dataset (Multi_Channel_Grid, abbreviated as MC_Grid) used in our paper [LIMUSE: LIGHTWEIGHT MULTI-MODAL SPEAKER EXTRACTION]([2111.04063] LiMuSE: Lightweight Multi-modal Speaker Extraction (arxiv.org)). MC_Grid, which is based on GRID dataset, includes multi-channel audio, extracted voiceprint and visual feature. The method of feature extraction will be introduced below. MC_Grid is specially prepared for speaker extraction task, and our code is available at aispeech-lab/LiMuSE. Feel free to contact us if you have any questions or suggestions.

We present a multilingual test set for conducting speech intelligibility tests in the form of diagnostic rhyme tests. The materials currently contain audio recordings in 5 languages and further extensions are in progress. For Mandarin Chinese, we provide recordings for a consonant contrast test as well as a tonal contrast test. Further information on the audio data, test procedure and software to set up a full survey which can be deployed on crowdsourcing platforms is provided in our paper [arXiv preprint] and GitHub repository. We welcome contributions to this open-source project.

WHAMR_ext is an extension to the WHAMR corpus with larger RT60 values (between 1s and 3s)