데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.
Mathematical dataset containing mathematical texts, i.e., texts containing LaTeX formulas, based on the AMPS Khan dataset and the ARQMath dataset V1.3. Based on the retrieved LaTeX texts, more mathematically equivalent versions have been generated by applying randomized LaTeX printing with this SymPy fork. A positive id corresponds to the ARQMath post id of the generated text version, a negative id indicates an AMPS text.
This dataset contains 3.3K expert-level pairwise human preferences for model responses generated by 6 models in response to 80 MT-bench questions. The 6 models are GPT-4, GPT-3.5, Claud-v1, Vicuna-13B, Alpaca-13B, and LLaMA-13B. The annotators are mostly graduate students with expertise in the topic areas of each of the questions.
The DukeMTMC-reID (Duke Multi-Tracking Multi-Camera ReIDentification) dataset is a subset of the DukeMTMC for image-based person re-ID. The dataset is created from high-resolution videos from 8 different cameras. It is one of the largest pedestrian image datasets wherein images are cropped by hand-drawn bounding boxes. The dataset consists 16,522 training images of 702 identities, 2,228 query images of the other 702 identities and 17,661 gallery images. NOTE: This dataset has been retracted. Source: Deep Co-attention based Comparators For Relative Representation Learning in Person Re-identification
WMT 2014 is a collection of datasets used in shared tasks of the Ninth Workshop on Statistical Machine Translation. The workshop featured four tasks: * a news translation task, * a quality estimation task, * a metrics task, * a medical text translation task. Source: https://www.aclweb.org/anthology/W14-3302.pdf
MSMT17 is a multi-scene multi-time person re-identification dataset. The dataset consists of 180 hours of videos, captured by 12 outdoor cameras, 3 indoor cameras, and during 12 time slots. The videos cover a long period of time and present complex lighting variations, and it contains a large number of annotated identities, i.e., 4,101 identities and 126,441 bounding boxes. The dataset can be requested at http://www.pkuvmc.com/dataset.html
WMT 2016 is a collection of datasets used in shared tasks of the First Conference on Machine Translation. The conference builds on ten previous Workshops on statistical Machine Translation. The conference featured ten shared tasks: - a news translation task, - an IT domain translation task, - a biomedical translation task, - an automatic post-editing task, - a metrics task (assess MT quality given reference translation). - a quality estimation task (assess MT quality without access to any reference), - a tuning task (optimize a given MT system), - a pronoun translation task, - a bilingual document alignment task, - a multimodal translation task. Source: http://www.statmt.org/wmt16/index.html
MTEB is a benchmark that spans 8 embedding tasks covering a total of 56 datasets and 112 languages. The 8 task types are Bitext mining, Classification, Clustering, Pair Classification, Reranking, Retrieval, Semantic Textual Similarity and Summarisation. The 56 datasets contain varying text lengths and they are grouped into three categories: Sentence to sentence, Paragraph to paragraph, and Sentence to paragraph. Check the latest leaderboards at HuggingFace.
We introduce here a new database called UBFC-rPPG (stands for Univ. Bourgogne Franche-Comté Remote PhotoPlethysmoGraphy) comprising two datasets that are focused specifically on rPPG analysis. The UBFC-RPPG database was created using a custom C++ application for video acquisition with a simple low cost webcam (Logitech C920 HD Pro) at 30fps with a resolution of 640x480 in uncompressed 8-bit RGB format. A CMS50E transmissive pulse oximeter was used to obtain the ground truth PPG data comprising the PPG waveform as well as the PPG heart rates. During the recording, the subject sits in front of the camera (about 1m away from the camera) with his/her face visible. All experiments are conducted indoors with a varying amount of sunlight and indoor illumination. The link to download the complete video dataset is available on request. A basic Matlab implementation can also be provided to read ground truth data acquired with a pulse oximeter.
A multilingual task-oriented semantic parsing dataset covering 6 languages and 11 domains.
The Machine Translation of Noisy Text (MTNT) dataset is a Machine Translation dataset that consists of noisy comments on Reddit and professionally sourced translation. The translation are between French, Japanese and French, with between 7k and 37k sentence per language pair. Source: https://arxiv.org/abs/1809.00388 Image Source: https://github.com/pmichel31415/mtnt
FLoRes is a benchmark dataset for machine translation between English and four low-resource languages, Nepali, Sinhala, Khmer, and Pashto, based on sentences translated from Wikipedia. The FLoRes project has two versions: ** FLoRes-101** and ** FLoRes-200**. - ** FLoRes-101**: This was the first version of the dataset. It allowed researchers to measure the quality of translations through 10,100 different translation directions. - ** FLoRes-200**: This is an updated version of the dataset. It doubles the existing language coverage of FLoRes-101. Given the nature of the new languages, which have less standardization and require more specialized professional translations, the verification process became more complex.
The DukeMTMC-VideoReID (Duke Multi-Tracking Multi-Camera Video-based ReIDentification) dataset is a subset of the DukeMTMC for video-based person re-ID. The dataset is created from high-resolution videos from 8 different cameras. It is one of the largest pedestrian video datasets wherein images are cropped by hand-drawn bounding boxes. The dataset consists 4832 tracklets of 1812 identities in total, and each tracklet has 168 frames on average. NOTE: This dataset has been retracted. Source: Exploit the Unknown Gradually: One-Shot Video-Based Person Re-Identification by Stepwise Learning
The OPUS-MT benchmark is a systematic collection of results from these models, focusing on verifiable translation performance and large coverage in terms of languages and domains. The OPUS-MT Dashboard is a web-based platform that provides a comprehensive overview of these open translation models. It includes summaries of benchmarks for over 2,300 models covering 4,560 language directions and 294 languages. The aim is to centralize, reproduce, and cover MT evaluation combined with scalability.
Occluded-DukeMTMC contains 15,618 training images, 17,661 gallery images, and 2,210 occluded query images. The experiment results on Occluded-DukeMTMC will demonstrate the superiority of our method in Occluded Person Re-ID problems, let alone that our method does not need any manually cropping procedure as pre-process.
The MTG-Jamendo dataset is an open dataset for music auto-tagging. The dataset contains over 55,000 full audio tracks with 195 tags categories (87 genre tags, 40 instrument tags, and 56 mood/theme tags). It is built using music available at Jamendo under Creative Commons licenses and tags provided by content uploaders. All audio is distributed in 320kbps MP3 format. A subset of the dataset is used in the Emotion and Theme Recognition in Music Task within MediaEval 2019. Source: https://mtg.github.io/mtg-jamendo-dataset/ Audio Source: https://essentia.upf.edu/datasets/mtg-jamendo/raw_30s/audio/
WMT 2018 is a collection of datasets used in shared tasks of the Third Conference on Machine Translation. The conference builds on a series of twelve previous annual workshops and conferences on Statistical Machine Translation. The conference featured ten shared tasks: - a news translation task, - a biomedical translation task, - a multimodal machine translation task, - a metrics task, - a quality estimation task, - an automatic post-editing task, - a parallel corpus filtering task. Source: http://www.statmt.org/wmt18/
A new multitask action quality assessment (AQA) dataset, the largest to date, comprising of more than 1600 diving samples; contains detailed annotations for fine-grained action recognition, commentary generation, and estimating the AQA score. Videos from multiple angles provided wherever available. Source: What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment
WMT 2015 is a collection of datasets used in shared tasks of the Tenth Workshop on Statistical Machine Translation. The workshop featured five tasks: * a news translation task, * a metrics task, * a tuning task, * a quality estimation task, * an automatic post-editing task. Source: https://www.aclweb.org/anthology/W15-3001.pdf
WMT 2020 is a collection of datasets used in shared tasks of the Fifth Conference on Machine Translation. The conference builds on a series of annual workshops and conferences on Statistical Machine Translation. The conference featured ten shared tasks: * a news translation task, * a biomedical translation task, * a similar language translation task, * an unsupervised and very low resource translation task, * an automatic post-editing task, * a metrics task (assess MT quality given reference translation), * a quality estimation task (assess MT quality without access to any reference), * a parallel corpus filtering and alignment task, * a lifelong learning in MT task, * a chat translation task. Source: https://www.statmt.org/wmt20/
News translation is a recurring WMT task. The test set is a collection of parallel corpora consisting of about 1500 English sentences translated into 5 languages (Czech, German, Finnish, Romanian, Russian, Turkish) and additional 1500 sentences from each of the 5 languages translated to English. For Romanian a third of the test set were released as a development set instead. For Turkish additional 500 sentence development set was released. The sentences were selected from dozens of news websites and translated by professional translators. The training data consists of parallel corpora to train translation models, monolingual corpora to train language models and development sets for tuning. Some training corpora were identical from WMT 2015 (Europarl, United Nations, French-English 10⁹ corpus, Common Crawl, Russian-English parallel data provided by Yandex, Wikipedia Headlines provided by CMU) and some were update (CzEng v1.6pre, News Commentary v11, monolingual news data). Additionally, the following new corpora were added: Romanian Europarl, SETIMES2 from OPUS for Romanian-English and Turkish-English, Monolingual data sets from CommonCrawl. Source: https://paperswithcode.com/paper/findings-of-the-2016-conference-on-machine/ Image Source: https://www.aclweb.org/anthology/W16-2301.pdf
MMT-Bench is a comprehensive benchmark designed to evaluate Large Vision-Language Models (LVLMs) across a wide array of multimodal tasks that require expert knowledge as well as deliberate visual recognition, localization, reasoning, and planning¹. It includes 31,325 meticulously curated multi-choice visual questions from various scenarios such as vehicle driving and embodied navigation, covering 32 core meta-tasks and 162 subtasks in multimodal understanding¹. The benchmark aims to be comprehensive enough to assess the multitask performance of LVLMs using a task map, which helps in identifying both in-domain and out-of-domain tasks. This extensive coverage allows for a thorough evaluation of the models' capabilities in multimodal understanding¹. (1) MMT-Bench. https://mmt-bench.github.io/. (2) OpenGVLab/MMT-Bench: ICML'2024 - GitHub. https://github.com/OpenGVLab/MMT-Bench. (3) OpenGVLab/MMT-Bench - Giters. https://giters.com/OpenGVLab/MMT-Bench.
The EPRSTMT dataset, also known as EPR-sentiment, is a binary sentiment analysis dataset based on product reviews on an e-commerce platform. Each sample in the dataset is labeled as either Positive or Negative. It was collected by the ICIP Lab of Beijing Normal University and has been re-organized to make it suitable for sentiment analysis tasks.
PARANMT-50M is a dataset for training paraphrastic sentence embeddings. It consists of more than 50 million English-English sentential paraphrase pairs. Source: ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations
P-DukeMTMC-reID is a modified version based on DukeMTMC-reID dataset. There are 12,927 images (665 identifies) in training set, 2,163 images (634 identities) for querying and 9,053 images in the gallery set.
News translation is a recurring WMT task. The test set is a collection of parallel corpora consisting of about 1500 English sentences translated into 5 languages (Chinese, Czech, Estonian, German, Finnish, Russian, Turkish) and additional 1500 sentences from each of the 7 languages translated to English. The sentences were selected from dozens of news websites and translated by professional translators. The training data consists of parallel corpora to train translation models, monolingual corpora to train language models and development sets for tuning. Some training corpora were identical from WMT 2017 (Europarl, Common Crawl, SETIMES2, Russian-English parallel data provided by Yandex, Wikipedia Headlines provided by CMU) and some were update (United Nations, CzEng v1.7, News Commentary v13, monolingual news data). Additionally, the EU Press Release parallel corpus for German, Finnish and Estonian was added. Source: https://www.statmt.org/wmt18/translation-task.html
To benchmark Bengali digit recognition algorithms, a large publicly available dataset is required which is free from biases originating from geographical location, gender, and age. With this aim in mind, NumtaDB, a dataset consisting of more than 85,000 images of hand-written Bengali digits, has been assembled.
PhoMT is a high-quality and large-scale Vietnamese-English parallel dataset of 3.02M sentence pairs for machine translation.
MSMT17-C is an evaluation set that consists of algorithmically generated corruptions applied to the MSMT17 test-set. These corruptions consist of Noise: Gaussian, shot, impulse, and speckle; Blur: defocus, frosted glass, motion, zoom, and Gaussian; Weather: snow, frost, fog, brightness, spatter, and rain; Digital: contrast, elastic, pixel, JPEG compression, and saturate. Each corruption has five severity levels, resulting in 100 distinct corruptions.
mTVR is a large-scale multilingual video moment retrieval dataset, containing 218K English and Chinese queries from 21.8K TV show video clips. The dataset is collected by extending the popular TVR dataset (in English) with paired Chinese queries and subtitles. Compared to existing moment retrieval datasets, mTVR is multilingual, larger, and comes with diverse annotations. Source: MTVR: Multilingual Moment Retrieval in Videos Image source: MTVR: Multilingual Moment Retrieval in Videos
The CommonMT is a dataset used for evaluating commonsense reasoning in neural machine translation. This dataset contains three types of test suites: - Lexical ambiguity - Contextless syntactic ambiguity - Contextual syntactic ambiguity
The images in DukeMTMC-attribute dataset comes from Duke University. There are 1812 identities and 34183 annotated bounding boxes in the DukeMTMC-attribute dataset. This dataset contains 702 identities for training and 1110 identities for testing, corresponding to 16522 and 17661 images respectively. The attributes are annotated in the identity level, every image in this dataset is annotated with 23 attributes. NOTE: This dataset has been retracted. Source: Pedestrian Attribute Recognition: A Survey Image Source: http://irip.buaa.edu.cn/mars_duke_attributes/index.html
FRMT is a dataset and evaluation benchmark for Few-shot Region-aware Machine Translation, a type of style-targeted translation. The dataset consists of human translations of a few thousand English Wikipedia sentences into regional variants of Portuguese and Mandarin. Source documents are selected to enable detailed analysis of phenomena of interest, including lexically distinct terms and distractor terms. Source:
MMToM-QA is the first multimodal benchmark to evaluate machine Theory of Mind (ToM), the ability to understand people's minds. MMToM-QA consists of 600 questions. Each question is paired with a clip of the full activity in a video (as RGB-D frames), as well as a text description of the scene and the actions taken by the person in that clip. All questions have two choices. The questions are categorized into seven types, evaluating belief inference and goal inference in rich and diverse situations. Each belief inference type has 100 questions, totaling 300 belief questions; each goal inference type has 75 questions, totaling 300 goal questions. The questions are paired with 134 videos of a person looking for daily objects in household environments.
MTASS is an open-source dataset in which mixtures contain three types of audio signals.
The expansion of social networks has accelerated the transmission of information and news at every communities. Over the past few years, the number of users, audiences and social networking publishers, are increased dramatically too. Among the massive amounts of information and news reported on these networks, we are faced with issues that have not been verified which is called “rumors”. Identifying rumors on social networks is carried out in the form of rumor detection approaches; the massive amount of these news and information force to use the machine learning techniques. The most important problem with auto-detection approaches is the lack of a database of rumors. For that matter, in this article, a collection of rumors published on the social network “telegrams” have been collected. These data are gathered from five Persian-language channels that have specially reviewed this issue. The collected data set contains 3283 messages with 2829 attachments, having a volume of over 1.6 gigabytes. This dataset can also be used for different purposes of natural language processing.
DivEMT, the first publicly available post-editing study of Neural Machine Translation (NMT) over a typologically diverse set of target languages. Using a strictly controlled setup, 18 professional translators were instructed to translate or post-edit the same set of English documents into Arabic, Dutch, Italian, Turkish, Ukrainian, and Vietnamese. During the process, their edits, keystrokes, editing times and pauses were recorded, enabling an in-depth, cross-lingual evaluation of NMT quality and post-editing effectiveness. Using this new dataset, we assess the impact of two state-of-the-art NMT systems, Google Translate and the multilingual mBART-50 model, on translation productivity.
FakeNewsAMT & Celebrity include two novel datasets for the task of fake news detection, covering seven different news domains.
The GitTables-SemTab dataset is a subset of the GitTables dataset and was created to be used during the SemTab challenge. The dataset consists of 1101 tables and is used to benchmark the Column Type Annotation (CTA) task. Its columns were annotated using semantic properties from DBpedia and semantic types and properties from Schema.org. The table below shows the number of annotated columns and number of classes used to annotate this dataset. | | Columns | Classes | |----------------|-------|-------| | Column Type Annotation - Schema.org | 721 | 59 | | Column Type Annotation - DBpedia | 2,533 | 122 |
This dataset contains time-stamped user retweet event sequences. The events are categorized into 3 types: retweets by “small,” “medium” and “large” users. Small users have fewer than 120 followers, medium users have fewer than 1363, and the rest are large users.
This shared task will examine automatic evaluation metrics for machine translation. The goals of the shared metrics task are: To achieve the strongest correlation with human judgement of translation quality; To illustrate the suitability of an automatic evaluation metric as a surrogate for human evaluation; To address problems associated with comparison with a single reference translation; To move automatic evaluation beyond system-level ranking to finer-grained sentence-level ranking. All datasets for this task are available here.
The dataset contains historical financial transactions, including time, category and cost fields. There are 50000 clients, 205 categories and 43.7M events. The original goal was to predict the age group of the client. In this variant of the dataset, the goal is to forecast multiple future events.
The dataset includes time-stamped user product reviews behavior from January, 2008 to October, 2018. Each user has a sequence of produce review events with each event containing the timestamp and category of the reviewed product, with each category corresponding to an event type.
Fashion-MNT is large-scale bilingual product description dataset called Fashion-MMT, which contains over 114k noisy and 40k manually cleaned description translations with multiple product images.
the MTHS dataset contains 30Hz PPG signals obtained from 62 patients, including 35 men and 27 women. The ground truth data includes heart rate and oxygen saturation levels sampled at 1Hz. The HR and SPo2 measurement is obtained using a pulse oximeter (M70). An iPhone 5s was used to obtain the ppg recordings at 30 fps.
The Mobile Turkish Scene Text (MTST 200) dataset consists of 200 indoor and outdoor Turkish scene text images. The images were collected with mobile phones and downsized to 576 × 1024 (portrait) or 1024 × 576 (landscape) pixels. The text lines are horizontal or near horizontal, some with slight in– and out-of-plane rotations.
The dataset has two years of user awards on a question-answering website: each user received a sequence of badges and there are 22 different kinds of badges in total.
This dataset comprises 1344 expert annotated images of muscle-tendon junctions recorded with 3 ultrasound imaging systems (Aixplorer V6, Esaote MyLab60, Telemed ArtUs), on 2 muscles (Lateral Gastrocnemius, Medial Gastrocnemius), and 2 movements (isometric maximum voluntary contractions, passive torque movements).
AMT Objects is a large dataset of object centric videos suitable for training and benchmarking models for generating 3D models of objects from a small number of photos of the objects. The dataset consists of multiple views of a large collection of object instances. The dataset contains 7 object categories from the MS COCO classes: apple, sandwich, orange, donut, banana, carrot and hydrant. For each class, annotators were asked to collect a video by looking ‘around’ a class instance, resulting in a turntable video. The dataset contains 169-457 videos per class. For each class, the videos were randomly split into training and testing videos in an 8:1 ratio.
The CICIoMT2024 dataset is a comprehensive dataset designed for cybersecurity research focused on the Internet of Medical Things (IoMT). Developed by the Canadian Institute for Cybersecurity, it simulates realistic IoMT network traffic, representing the diverse and evolving threats faced by connected healthcare devices. The dataset comprises network traffic data from various IoMT devices, with labeled instances for 18 distinct types of cyberattacks, alongside benign traffic data. Each cyberattack type is meticulously crafted to reflect common and advanced threats, such as Distributed Denial of Service (DDoS), ransomware, man-in-the-middle attacks, and malware injections. With a balanced mix of attacks and benign instances, CICIoMT2024 enables robust training and testing for various cybersecurity models. The CICIoMT2024 dataset’s detailed packet-level information and multi-class labels make it ideal for developing and evaluating machine learning and deep learning algorithms. Its design emphasizes real-world application, making it a valuable resource for researchers aiming to enhance IoMT security, protect patient data, and ensure the resilience of medical networks against cyber threats.
57 stock videos from Pexels, predominantly covering road scenes which involve minimal distortion. They involve different camera setups, also with varying camera heights, obstacles present throughout some videos (for e.g. car hood), highly varying image resolutions, and even weather conditions (day, rain, snow, night etc.). For more details, check the paper CamLessMonoDepth: Monocular Depth Estimation with Unknown Camera Parameters https://arxiv.org/pdf/2110.14347v1.pdf