Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 25편

MULTEXT-East

MULTEXT-East 설명 보기

The MULTEXT-East resources are a multilingual dataset for language engineering research and development. It consists of the (1) MULTEXT-East morphosyntactic specifications, defining categories (parts-of-speech), their morphosyntactic features (attributes and values), and the compact MSD tagset representations; (2) morphosyntactic lexica, (3) the annotated parallel "1984" corpus; and (4) some comparable text and speech corpora. The specifications are available for the following macrolanguages, languages and language varieties: Albanian, Bulgarian, Chechen, Czech, Damaskini, English, Estonian, Hungarian, Macedonian, Persian, Polish, Resian, Romanian, Russian, Serbo-Croatian, Slovak, Slovene, Torlak, and Ukrainian, while the other resources are available for a subset of these languages.

관련 논문 25편

MotSynth

MotSynth 설명 보기

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

관련 논문 25편

OpenAssistant Conversations Dataset

OASST1
OASST1 설명 보기

license: apache-2.0 tags: human-feedback size_categories: 100K<n<1M pretty_name: OpenAssistant Conversations # OpenAssistant Conversations Dataset (OASST1) ## Dataset Description - Homepage: https://www.open-assistant.io/ - Repository: https://github.com/LAION-AI/Open-Assistant - Paper: https://arxiv.org/abs/2304.07327 ### Dataset Summary In an effort to democratize research on large-scale alignment, we release OpenAssistant Conversations (OASST1), a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages in 35 different languages, annotated with 461,292 quality ratings, resulting in over 10,000 fully annotated conversation trees. The corpus is a product of a worldwide crowd-sourcing effort involving over 13,500 volunteers. Please refer to our paper for further details. ### Dataset Structure This dataset contains message trees. Each message tree has an initial prompt message as the root node, which can have multiple child messages as replies, and these child messages can have multiple replies. All messages have a role property: this can either be "assistant" or "prompter". The roles in conversation threads from prompt to leaf node strictly alternate between "prompter" and "assistant". This version of the dataset contains data collected on the open-assistant.io website until April 12 2023. ### JSON Example: Message For readability, the following JSON examples are shown formatted with indentation on multiple lines. Objects are stored without indentation (on single lines) in the actual jsonl files. json { "message_id": "218440fd-5317-4355-91dc-d001416df62b", "parent_id": "13592dfb-a6f9-4748-a92c-32b34e239bb4", "user_id": "8e95461f-5e94-4d8b-a2fb-d4717ce973e4", "text": "It was the winter of 2035, and artificial intelligence (..)", "role": "assistant", "lang": "en", "review_count": 3, "review_result": true, "deleted": false, "rank": 0, "synthetic": true, "model_name": "oasst-sft-0_3000,max_new_tokens=400 (..)", "labels": { "spam": { "value": 0.0, "count": 3 }, "lang_mismatch": { "value": 0.0, "count": 3 }, "pii": { "value": 0.0, "count": 3 }, "not_appropriate": { "value": 0.0, "count": 3 }, "hate_speech": { "value": 0.0, "count": 3 }, "sexual_content": { "value": 0.0, "count": 3 }, "quality": { "value": 0.416, "count": 3 }, "toxicity": { "value": 0.16, "count": 3 }, "humor": { "value": 0.0, "count": 3 }, "creativity": { "value": 0.33, "count": 3 }, "violence": { "value": 0.16, "count": 3 } } } ### JSON Example: Conversation Tree For readability, only a subset of the message properties is shown here. json { "message_tree_id": "14fbb664-a620-45ce-bee4-7c519b16a793", "tree_state": "ready_for_export", "prompt": { "message_id": "14fbb664-a620-45ce-bee4-7c519b16a793", "text": "Why can't we divide by 0? (..)", "role": "prompter", "lang": "en", "replies": [ { "message_id": "894d30b6-56b4-4605-a504-89dd15d4d1c8", "text": "The reason we cannot divide by zero is because (..)", "role": "assistant", "lang": "en", "replies": [ // ... ] }, { "message_id": "84d0913b-0fd9-4508-8ef5-205626a7039d", "text": "The reason that the result of a division by zero is (..)", "role": "assistant", "lang": "en", "replies": [ { "message_id": "3352725e-f424-4e3b-a627-b6db831bdbaa", "text": "Math is confusing. Like those weird Irrational (..)", "role": "prompter", "lang": "en", "replies": [ { "message_id": "f46207ca-3149-46e9-a466-9163d4ce499c", "text": "Irrational numbers are simply numbers (..)", "role": "assistant", "lang": "en", "replies": [] }, // ... ] } ] } ] } } Please refer to oasst-data for details about the data structure and Python code to read and write jsonl files containing oasst data objects. If you would like to explore the dataset yourself you can find a getting-started notebook in the notebooks/openassistant-oasst1 folder of the LAION-AI/Open-Assistant github repository. ## Main Dataset Files Conversation data is provided either as nested messages in trees (extension .trees.jsonl.gz) or as a flat list (table) of messages (extension .messages.jsonl.gz). ### Ready For Export Trees 2023-04-12_oasst_ready.trees.jsonl.gz 10,364 trees with 88,838 total messages 2023-04-12_oasst_ready.messages.jsonl.gz 88,838 messages Trees in ready_for_export state without spam and deleted messages including message labels. The oasst_ready-trees file usually is sufficient for supervised fine-tuning (SFT) & reward model (RM) training. ### All Trees 2023-04-12_oasst_all.trees.jsonl.gz 66,497 trees with 161,443 total messages 2023-04-12_oasst_all.messages.jsonl.gz 161,443 messages All trees, including those in states prompt_lottery_waiting (trees that consist of only one message, namely the initial prompt), aborted_low_grade (trees that stopped growing because the messages had low quality), and halted_by_moderator. ### Supplemental Exports: Spam & Prompts 2023-04-12_oasst_spam.messages.jsonl.gz These are messages which were deleted or have a negative review result ("review_result": false). Besides low quality, a frequent reason for message deletion is a wrong language tag. 2023-04-12_oasst_prompts.messages.jsonl.gz These are all the kept initial prompt messages with positive review result (no spam) of trees in ready_for_export or prompt_lottery_waiting state. ### Using the Huggingface Datasets While HF datasets is ideal for tabular datasets, it is not a natural fit for nested data structures like the OpenAssistant conversation trees. Nevertheless, we make all messages which can also be found in the file 2023-04-12_oasst_ready.trees.jsonl.gz available in parquet as train/validation splits. These are directly loadable by Huggingface Datasets. To load the oasst1 train & validation splits use: python from datasets import load_dataset ds = load_dataset("OpenAssistant/oasst1") train = ds['train'] # len(train)=84437 (95%) val = ds['validation'] # len(val)=4401 (5%) The messages appear in depth-first order of the message trees. Full conversation trees can be reconstructed from the flat messages table by using the parent_id and message_id properties to identify the parent-child relationship of messages. The message_tree_id and tree_state properties (only present in flat messages files) can be used to find all messages of a message tree or to select trees by their state. ### Languages OpenAssistant Conversations incorporates 35 different languages with a distribution of messages as follows: Languages with over 1000 messages - English: 71956 - Spanish: 43061 - Russian: 9089 - German: 5279 - Chinese: 4962 - French: 4251 - Thai: 3042 - Portuguese (Brazil): 2969 - Catalan: 2260 - Korean: 1553 - Ukrainian: 1352 - Italian: 1320 - Japanese: 1018 <details> <summary><b>Languages with under 1000 messages</b></summary> <ul> <li>Vietnamese: 952</li> <li>Basque: 947</li> <li>Polish: 886</li> <li>Hungarian: 811</li> <li>Arabic: 666</li> <li>Dutch: 628</li> <li>Swedish: 512</li> <li>Turkish: 454</li> <li>Finnish: 386</li> <li>Czech: 372</li> <li>Danish: 358</li> <li>Galician: 339</li> <li>Hebrew: 255</li> <li>Romanian: 200</li> <li>Norwegian Bokmål: 133</li> <li>Indonesian: 115</li> <li>Bulgarian: 95</li> <li>Bengali: 82</li> <li>Persian: 72</li> <li>Greek: 66</li> <li>Esperanto: 59</li> <li>Slovak: 19</li> </ul> </details> ## Contact - Discord Open Assistant Discord Server - GitHub: LAION-AI/Open-Assistant - E-Mail: open-assistant@laion.ai

관련 논문 25편

RWTH-PHOENIX-Weather 2014

RWTH-PHOENIX-Weather 2014 설명 보기

The signing is recorded by a stationary color camera placed in front of the sign language interpreters. Interpreters wear dark clothes in front of an artificial grey background with color transition. All recorded videos are at 25 frames per second and the size of the frames is 210 by 260 pixels. Each frame shows the interpreter box only.

관련 논문 25편

ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes

ScanNet++
ScanNet++ 설명 보기

ScanNet++ is a large scale dataset with 450+ 3D indoor scenes containing sub-millimeter resolution laser scans, registered 33-megapixel DSLR images, and commodity RGB-D streams from iPhone. The 3D reconstructions are annotated with long-tail and label-ambiguous semantics to benchmark semantic understanding methods, while the coupled DSLR and iPhone captures enable benchmarking of novel view synthesis methods in high-quality and commodity settings.

관련 논문 25편

Task Oriented Parsing v2

TOPv2
TOPv2 설명 보기

Task Oriented Parsing v2 (TOPv2) representations for intent-slot based dialog systems. Provided under the CC-BY-SA license. Please cite the accompanying paper when using this dataset - @inproceedings{chen-etal-2020-low-resource, title={Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing}, author={Xilun Chen and Asish Ghoshal and Yashar Mehdad and Luke Zettlemoyer and Sonal Gupta}, booktitle={Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)}, year={2020}, publisher = "Association for Computational Linguistics" } CHANGELOG:<br> 03/10/2021 (V1.1): Added the low-resource splits used in the paper.<br> 09/18/2020 (V1.0): Initial release. TOPv2 is a multi-domain task-oriented semantic parsing dataset. It is an extension to the TOP dataset (http://fb.me/semanticparsingdialog) with 6 additional domains and 137k new samples. In total, TOPv2 has 8 domains (alarm, event, messaging, music, navigation, reminder, timer, weather) and 180k samples randomly split into train, eval, and test sets for each domain. Please refer to the paper for more data statistics. Note: As TOPv2 data is provided on a per-domain basis, the UNSUPPORTED utterances in the original TOP dataset were removed as they could not be mapped to any domain. The training, evaluation and test sets for each domain are provided as tab-separated value (TSV) files with file names of "domain_split.tsv". The first row of each file contains the column headers, while each following row is of the format: domain <tab> utterance <tab> semantic_parse where the semantic_parse follows the same format as the original TOP dataset. e.g. event <tab> Art fairs this weekend in Detroit <tab> [IN:GET_EVENT [SL:CATEGORY_EVENT Art fairs ] [SL:DATE_TIME this weekend ] in [SL:LOCATION Detroit ] ] The low-resource splits used in our experiments are provided in the low_resource_splits subdirectory, including training and validation sets from the reminder and weather domains under 10, 25, 50, 100, 250, 500 and 1000 SPIS.

관련 논문 25편

TinyPerson

TinyPerson 설명 보기

TinyPerson is a benchmark for tiny object detection in a long distance and with massive backgrounds. The images in TinyPerson are collected from the Internet. First, videos with a high resolution are collected from different websites. Second, images from the video are sampled every 50 frames. Then images with a certain repetition (homogeneity) are deleted, and the resulting images are annotated with 72,651 objects with bounding boxes by hand. Source: https://arxiv.org/abs/1912.10664 Image Source: https://arxiv.org/pdf/1912.10664.pdf

관련 논문 25편

ZESHEL

ZESHEL 설명 보기

ZESHEL is a zero-shot entity linking dataset, which places more emphasis on understanding the unstructured descriptions of entities to resolve the ambiguity of mentions on four unseen domains. This dataset was constructed using Wikias from FANDOM.

관련 논문 24편

105,941 Images Natural Scenes OCR Data of 12 Languages

105,941 Images Natural Scenes OCR Data of 12 Languages 설명 보기

Description: 105,941 Images Natural Scenes OCR Data of 12 Languages. The data covers 12 languages (6 Asian languages, 6 European languages), multiple natural scenes, multiple photographic angles. For annotation, line-level quadrilateral bounding box annotation and transcription for the texts were annotated in the data. The data can be used for tasks such as OCR of multi-language. Data size: 105,941 images, including Asian language family: Japanese 9,997 images, Korean 10,231 images, Indonesian 7,591 images, Malay 5,650 images, Vietnamese 8,822 images, Thai 9,645 images; European language family: French 10,015 images, German 7,213 images, Italian 8,824 images, Portuguese 7,754 images, Russian 10,376 images and Spanish 9,823 images Collecting environment: including shop plaque, stop board, poster, ticket, road sign, comic, cover picture, prompt/reminder, warning, packing instruction, menu, building sign, etc.

관련 논문 24편

The 19 landmark variant of AFLW.

AFLW-19
AFLW-19 설명 보기

The original AFLW provides at most 21 points for each face, but excluding coordinates for invisible landmarks, causing difficulties for training most of the existing baseline approaches. To make fair comparisons, the authors manually annotate the coordinates of these invisible landmarks to enable training of those baseline approaches. The new annotation does not include two ear points because it is very difficult to decide the location of invisible ears. This causes the point number of AFLW-19 to be 19. The original AFLW does not provide train-test partition. AFLW-19 adopts a partition with 20,000 images for training and 4,386 images for testing (AFLW-Full). In addition, a frontal subset (AFLW-Frontal) is proposed where all landmarks are visible (totally 1,165 images). The new 19-point annotation file is available at the project page.

관련 논문 24편

ARCD

ARCD 설명 보기

Composed of 1,395 questions posed by crowdworkers on Wikipedia articles, and a machine translation of the Stanford Question Answering Dataset (Arabic-SQuAD). Source: Neural Arabic Question Answering

관련 논문 24편

CASIA-SURF

CASIA-SURF 설명 보기

Dataset for face anti-spoofing in terms of both subjects and modalities. Specifically, it consists of subjects with videos and each sample has modalities (i.e., RGB, Depth and IR). Source: CASIA-SURF: A Large-scale Multi-modal Benchmark for Face Anti-spoofing

관련 논문 24편

DHF1K

DHF1K 설명 보기

DHF1K is a video saliency dataset which contains a ground-truth map of binary pixel-wise gaze fixation points and a continuous map of the fixation points after being blurred by a gaussian filter. DHF1K contains 1000 videos in total. 700 of the videos are annotated, 600 of which are used for training and 100 for validation. The remaining 300 are the testing set which are to be evaluated on a public server. Source: ViP: Video Platform for PyTorch Image Source: https://arxiv.org/pdf/1801.07424.pdf

관련 논문 24편

FineAction

FineAction 설명 보기

FineAction contains 103K temporal instances of 106 action categories, annotated in 17K untrimmed videos. FineAction introduces new opportunities and challenges for temporal action localization, thanks to its distinct characteristics of fine action classes with rich diversity, dense annotations of multiple instances, and co-occurring actions of different classes.

관련 논문 24편

FlickrStyle10K

FlickrStyle10K 설명 보기

FlickrStyle10K is collected and built on Flickr30K image caption dataset. The original FlickrStyle10K dataset has 10,000 pairs of images and stylized captions including humorous and romantic styles. However, only 7,000 pairs from the official training set are now publicly accessible. The dataset can be downloaded via https://zhegan27.github.io/Papers/FlickrStyle_v0.9.zip

관련 논문 24편

FreeSolv (Free Solvation)

FreeSolv (Free Solvation) 설명 보기

The FreeSolv database offers a curated collection of experimental and calculated hydration-free energies for small molecules in water. It includes both experimental values obtained from prior literature and calculated values based on simulations. The goal is to provide accurate hydration-free energy data, which is essential for understanding solvation properties and interactions of molecules in aqueous environments.

관련 논문 24편

GeoS

GeoS 설명 보기

GeoS is a dataset for automatic math problem solving. It is a dataset of SAT plane geometry questions where every question has a textual description in English accompanied by a diagram and multiple choices. Questions and answers are compiled from previous official SAT exams and practice exams offered by the College Board. We annotate ground-truth logical forms for all questions in the dataset.

관련 논문 24편

Google Landmarks

Google Landmarks 설명 보기

The Google Landmarks dataset contains 1,060,709 images from 12,894 landmarks, and 111,036 additional query images. The images in the dataset are captured at various locations in the world, and each image is associated with a GPS coordinate. This dataset is used to train and evaluate large-scale image retrieval models.

관련 논문 24편

KITTI-STEP

KITTI-STEP 설명 보기

The Segmenting and Tracking Every Pixel (STEP) benchmark consists of 21 training sequences and 29 test sequences. It is based on the KITTI Tracking Evaluation and the Multi-Object Tracking and Segmentation (MOTS) benchmark. This benchmark extends the annotations to the Segmenting and Tracking Every Pixel (STEP) task. [Copy-pasted from http://www.cvlibs.net/datasets/kitti/eval_step.php]

관련 논문 24편

MCScript

MCScript 설명 보기

MCScript is used as the official dataset of SemEval2018 Task11. This dataset constructs a collection of text passages about daily life activities and a series of questions referring to each passage, and each question is equipped with two answer choices. The MCScript comprises 9731, 1411, and 2797 questions in training, development, and test set respectively. Source: Multi-Perspective Fusion Network for Commonsense Reading Comprehension Image Source: https://arxiv.org/pdf/1803.05223.pdf

관련 논문 24편

Multimodal Spontaneous Expression-Heart Rate dataset

MMSE-HR
MMSE-HR 설명 보기

The MMSE-HR benchmark consists of a dataset of 102 videos from 40 subjects recorded at 1040x1392 raw resolution at 25fps. During the recordings, various stimuli such as videos, sounds, and smells are introduced to induce different emotional states in the subjects. The ground truth waveform for MMSE-HR is the blood pressure signal sampled at 1000Hz. The dataset contains a diverse distribution of skin colors in the Fitzpatrick scale (II=8, III=11, IV=17, V+VI=4).

관련 논문 24편

MSU Video Upscalers: Quality Enhancement

MSU Video Upscalers: Quality Enhancement 설명 보기

The dataset aims to find the algorithms that produce the most visually pleasant image possible and generalize well to a broad range of content. It consists of 30 clips and contains 15 2D-animated segments losslessly recorded from various video games and 15 camera-shot segments from high-bitrate YUV444 sources. The complexity of clips varies significantly in terms of spatial and temporal indexes. Multiple bicubic downscaling mixed with sharpening is used to simulate complex real-world camera degradation. The authors used slight compression and YUV420 conversion to simulate a practical use case. 1920×1080 sources were downscaled to 480×270 input.

관련 논문 24편

Moral Stories

Moral Stories 설명 보기

Moral Stories is a crowd-sourced dataset of structured narratives that describe normative and norm-divergent actions taken by individuals to accomplish certain intentions in concrete situations, and their respective consequences. Source: Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences

관련 논문 24편

Natural Stories

Natural Stories 설명 보기

The Natural Stories dataset consists of English texts edited to contain many low-frequency syntactic constructions while still sounding fluent to native speakers. The corpus is annotated with hand-corrected parse trees and includes self-paced reading time data. Source: The Natural Stories Corpus

관련 논문 24편

Okutama-Action

Okutama-Action 설명 보기

A new video dataset for aerial view concurrent human action detection. It consists of 43 minute-long fully-annotated sequences with 12 action classes. Okutama-Action features many challenges missing in current datasets, including dynamic transition of actions, significant changes in scale and aspect ratio, abrupt camera movement, as well as multi-labeled actors. Source: Okutama-Action: An Aerial View Video Dataset for Concurrent Human Action Detection

관련 논문 24편

QM7

QM7 설명 보기

QM7 dataset is a subset of the GDB-13 database. GDB-13 contains nearly 1 billion stable and synthetically accessible organic molecules. In the QM7 subset, only molecules with up to 23 atoms are included. These atoms consist of carbon ©, nitrogen (N), oxygen (O), and sulfur (S). The total number of molecules in the QM7 dataset is 7165. Each molecule is represented using the Coulomb matrix, which captures the interactions between atoms.

관련 논문 24편

RAdar Dataset In Adverse weaThEr

RADIATE
RADIATE 설명 보기

RADIATE (RAdar Dataset In Adverse weaThEr) is new automotive dataset created by Heriot-Watt University which includes Radar, Lidar, Stereo Camera and GPS/IMU. The data is collected in different weather scenarios (sunny, overcast, night, fog, rain and snow) to help the research community to develop new methods of vehicle perception. The radar images are annotated in 7 different scenarios: Sunny (Parked), Sunny/Overcast (Urban), Overcast (Motorway), Night (Motorway), Rain (Suburban), Fog (Suburban) and Snow (Suburban). The dataset contains 8 different types of objects (car, van, truck, bus, motorbike, bicycle, pedestrian and group of pedestrians). Source: https://github.com/marcelsheeny/radiate_sdk Image Source: https://github.com/marcelsheeny/radiate_sdk

관련 논문 24편

Region-aware benchmark based on the LYHM

REALY
REALY 설명 보기

The REALY benchmark aims to introduce a region-aware evaluation pipeline to measure the fine-grained normalized mean square error (NMSE) of 3D face reconstruction methods from under-controlled image sets. Given the reconstructed mesh from the 2D image in REALY by a specific method, the REALY benchmark calculates the similarity of ground-truth scans on four regions (nose, mouth, forehead, cheek) with the predicted mesh.

관련 논문 24편

REDDIT-12K

REDDIT-12K 설명 보기

Reddit12k contains 11929 graphs each corresponding to an online discussion thread where nodes represent users, and an edge represents the fact that one of the two users responded to the comment of the other user. There is 1 of 11 graph labels associated with each of these 11929 discussion graphs, representing the category of the community. Source: Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity

관련 논문 24편

Reasoning Over Paragraph Effects in Situations

ROPES
ROPES 설명 보기

ROPES is a QA dataset which tests a system's ability to apply knowledge from a passage of text to a new situation. A system is presented a background passage containing a causal or qualitative relation(s), a novel situation that uses this background, and questions that require reasoning about effects of the relationships in the back-ground passage in the context of the situation. Source: Allen Institute for AI

관련 논문 24편

RecipeQA

RecipeQA 설명 보기

RecipeQA is a dataset for multimodal comprehension of cooking recipes. It consists of over 36K question-answer pairs automatically generated from approximately 20K unique recipes with step-by-step instructions and images. Each question in RecipeQA involves multiple modalities such as titles, descriptions or images, and working towards an answer requires (i) joint understanding of images and text, (ii) capturing the temporal flow of events, and (iii) making sense of procedural knowledge. Source: RecipeQA

관련 논문 24편

Situated and Interactive Multimodal Conversations

SIMMC
SIMMC 설명 보기

Situated Interactive MultiModal Conversations (SIMMC) is the task of taking multimodal actions grounded in a co-evolving multimodal input content in addition to the dialog history. This dataset contains two SIMMC datasets totalling ~13K human-human dialogs (~169K utterances) using a multimodal Wizard-of-Oz (WoZ) setup, on two shopping domains: (a) furniture (grounded in a shared virtual environment) and (b) fashion (grounded in an evolving set of images). Source: https://github.com/facebookresearch/simmc Image Source: https://github.com/facebookresearch/simmc

관련 논문 24편

Stereo Quantitative Underwater Image Dataset

SQUID
SQUID 설명 보기

A dataset of images taken in different locations with varying water properties, showing color charts in the scenes. Moreover, to obtain ground truth, the 3D structure of the scene was calculated based on stereo imaging. This dataset enables a quantitative evaluation of restoration algorithms on natural images. Source: Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset

관련 논문 24편

Spoken-SQuAD

Spoken-SQuAD 설명 보기

In SpokenSQuAD, the document is in spoken form, the input question is in the form of text and the answer to each question is always a span in the document. The following procedures were used to generate spoken documents from the original SQuAD dataset. First, the Google text-to-speech system was used to generate the spoken version of the articles in SQuAD. Then CMU Sphinx was sued to generate the corresponding ASR transcriptions. The SQuAD training set was used to generate the training set of Spoken SQuAD, and SQuAD development set was used to generate the testing set for Spoken SQuAD. If the answer of a question did not exist in the ASR transcriptions of the associated article, the question-answer pair was removed from the dataset because these examples are too difficult for listening comprehension machine at this stage. Source: https://github.com/chiahsuan156/Spoken-SQuAD Image Source: https://github.com/chiahsuan156/Spoken-SQuAD

관련 논문 24편

The TechQA Dataset

TechQA
TechQA 설명 보기

TECHQA is a domain-adaptation question answering dataset for the technical support domain. The TECHQA corpus highlights two real-world issues from the automated customer support domain. First, it contains actual questions posed by users on a technical forum, rather than questions generated specifically for a competition or a task. Second, it has a real-world size – 600 training, 310 dev, and 490 evaluation question/answer pairs – thus reflecting the cost of creating large labeled datasets with actual data. Consequently, TECHQA is meant to stimulate research in domain adaptation rather than being a resource to build QA systems from scratch. The dataset was obtained by crawling the IBM Developer and IBM DeveloperWorks forums for questions with accepted answers that appear in a published IBM Technote—a technical document that addresses a specific technical issue. Source: The TechQA Dataset