Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 53편

MMVP

MMVP 설명 보기

The MMVP (Multimodal Visual Patterns) Benchmark focuses on identifying "CLIP-blind pairs" – images that appear similar to the CLIP model despite having clear visual differences. These patterns highlight the challenges these systems face in answering straightforward questions, often leading to incorrect responses and hallucinated explanations.

관련 논문 7편

MMVP-VLM

MMVP-VLM 설명 보기

The MMVP-VLM (Multimodal Visual Patterns - Visual Language Models) Benchmark is specifically designed to systematically evaluate the performance of recent CLIP-based models in understanding and processing visual patterns. Let's delve into the details: - Purpose: The MMVP-VLM Benchmark aims to assess how well CLIP models can match image-text combinations that represent distinct visual patterns. It distills a subset of questions from the original MMVP benchmark into simpler language descriptions, categorizing them into different visual patterns. Each visual pattern is represented by 15 text-image pairs. - Dataset Composition: - Text-Image Pairs: The benchmark includes a balanced number of questions for each visual pattern, with each pattern represented by 15 pairs. These pairs are a subset of the MMVP benchmark, supplemented with additional questions for balance. - Visual Patterns: The questions cover various visual patterns, allowing evaluation of CLIP models' ability to understand and process these patterns. - Insights and Limitations: By assessing whether CLIP models can accurately match the provided image-text combinations, the MMVP-VLM Benchmark provides insights into the capabilities and limitations of these models.