데이터셋 탐색

This dataset contains 3.3K expert-level pairwise human preferences for model responses generated by 6 models in response to 80 MT-bench questions. The 6 models are GPT-4, GPT-3.5, Claud-v1, Vicuna-13B, Alpaca-13B, and LLaMA-13B. The annotators are mostly graduate students with expertise in the topic areas of each of the questions.

MMT-Bench is a comprehensive benchmark designed to evaluate Large Vision-Language Models (LVLMs) across a wide array of multimodal tasks that require expert knowledge as well as deliberate visual recognition, localization, reasoning, and planning¹. It includes 31,325 meticulously curated multi-choice visual questions from various scenarios such as vehicle driving and embodied navigation, covering 32 core meta-tasks and 162 subtasks in multimodal understanding¹. The benchmark aims to be comprehensive enough to assess the multitask performance of LVLMs using a task map, which helps in identifying both in-domain and out-of-domain tasks. This extensive coverage allows for a thorough evaluation of the models' capabilities in multimodal understanding¹. (1) MMT-Bench. https://mmt-bench.github.io/. (2) OpenGVLab/MMT-Bench: ICML'2024 - GitHub. https://github.com/OpenGVLab/MMT-Bench. (3) OpenGVLab/MMT-Bench - Giters. https://giters.com/OpenGVLab/MMT-Bench.

MT-Bench in Thai.