Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 1편

An Enhanced MultiModal ReAsoning Benchmark

EMMA
EMMA 설명 보기

We introduce EMMA (Enhanced MultiModal reAsoning), a benchmark targeting organic multimodal reasoning across mathematics, physics, chemistry, and coding. EMMA tasks demand advanced cross-modal reasoning that cannot be solved by thinking separately in each modality, offering an enhanced test suite for MLLMs' reasoning capabilities. EMMA is composed of 2,788 problems, of which 1,796 are newly constructed, across four domains. Within each subject, we further provide fine-grained labels for each question based on the specific skills it measures.

관련 논문 2편

JEMMA

JEMMA 설명 보기

JEMMA is an Extensible Java Dataset for ML4Code Applications, which is a large-scale dataset targeted at ML4 code. JEMMA comes with a considerable amount of pre-processed information such as metadata, representations (e.g., code tokens, ASTs, graphs), and several properties (e.g., metrics, static analysis results) for 50,000 Java projects from the 50KC dataset, with over 1.2 million classes and over 8 million methods. Source: JEMMA: An Extensible Java Dataset for ML4Code Applications Image Source: https://arxiv.org/pdf/2212.09132v1.pdf

관련 논문 1편

LEMMA

LEMMA 설명 보기

The LEMMA dataset aims to explore the essence of complex human activities in a goal-directed, multi-agent, multi-task setting with ground-truth labels of compositional atomic-actions and their associated tasks. By quantifying the scenarios to up to two multi-step tasks with two agents, the authors strive to address human multi-task and multi-agent interactions in four scenarios: single-agent single-task (1 x 1), single-agent multi-task (1 x 2), multi-agent single-task (2 x 1), and multi-agent multi-task (2 x 2). Task instructions are only given to one agent in the 2 x 1 setting to resemble the robot-helping scenario, hoping that the learned perception models could be applied in robotic tasks (especially in HRI) in the near future. Both the third-person views (TPVs) and the first-person views (FPVs) were recorded to account for different perspectives of the same activities. The authors densely annotate atomic-actions (in the form of compositional verb-noun pairs) and tasks of each atomic-action, as well as the spatial location of each participating agent (bounding boxes) to facilitate the learning of multi-agent multi-task task scheduling and assignment. Source: LEMMA

관련 논문 1편

Long-video Sampling Dilemma Benchmark

LSDBench
LSDBench 설명 보기

A benchmark that focuses on the sampling dilemma in long-video tasks. The LSDBench dataset is designed to evaluate the sampling efficiency of long-video VLMs. It consists of multiple-choice question-answer pairs based on hour-long videos, focusing on dense and short-duration actions with high Necessary Sampling Density (NSD).

관련 논문 1편

Tuan Dung nguyen

Mapping Topics in 100,000 Real-Life Moral Dilemmas
Mapping Topics in 100,000 Real-Life Moral Dilemmas 설명 보기

This dataset accompanies the ICWSM 2022 paper "Mapping Topics in 100,000 Real-Life Moral Dilemmas".

관련 논문 1편

TemMat dataset

TemMat
TemMat 설명 보기

Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset

관련 논문 1편

The Trilemma of Truth in Large Language Models

Trilemma Dataset
Trilemma Dataset 설명 보기

The Trilemma of Truth is a multiclass probing dataset for evaluating the veracity-tracking mechanism of large language models. It comprises three configurations— city_locations (statements about where cities are), med_indications (drug–condition assignments), and word_definitions - each annotated as true, false, or neither. Designed to plug straight into sAwMIL and various veracity probes.