WebVidVQA3M
A dataset automatically generated using question generation neural models and alt-text video captions from the WebVid dataset, with 3M video-question-answer triplets.
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
Visual Question Answering (VQA) is a task in computer vision that involves answering questions about an image. The goal of VQA is to teach machines to understand the content of an image and answer questions about it in natural language.
Image Source: visualqa.org
출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.
A dataset automatically generated using question generation neural models and alt-text video captions from the WebVid dataset, with 3M video-question-answer triplets.
Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data. The dataset can be found in https://huggingface.co/datasets/worldcuisines/vqa
Click to add a brief description of the dataset (Markdown and LaTeX enabled). Provide: * a high-level explanation of the dataset characteristics * explain motivations and summary of its content * potential use cases of the dataset
The OpenEQA dataset is a significant contribution in the field of Embodied Question Answering (EQA). Let me provide you with some details: 1. Definition: - Embodied Question Answering (EQA) involves understanding an environment well enough to answer questions about it in natural language. - EQA agents can achieve this understanding through either episodic memory (as seen in agents using smart glasses) or active exploration of the environment (as in the case of mobile robots). 2. OpenEQA Dataset: - OpenEQA is the first open-vocabulary benchmark dataset for EQA that supports both episodic memory and active exploration use cases. - It contains over 1600 high-quality human-generated questions drawn from more than 180 real-world environments. - The dataset consists of question-answer pairs ($Q, A^*$) and corresponding episode histories ($H$). - You can find the question-answer pairs in the file data/open-eqa-v0.json. - To access the episode histories, follow the instructions provided here. 3. Evaluation Protocol: - OpenEQA also provides an automatic evaluation protocol powered by language model-based evaluation (LLM). - This evaluation protocol correlates well with human judgment. 4. Foundation Models Evaluation: - Researchers evaluated several state-of-the-art foundation models, including GPT-4V, using the OpenEQA dataset. - The findings revealed that these models significantly lag behind human-level performance in EQA tasks. 5. Significance: - OpenEQA serves as a straightforward, measurable, and practically relevant benchmark for current-generation foundation models. - It poses a considerable challenge and inspires research at the intersection of Embodied AI, conversational agents, and world models.
RealWorldQA is a benchmark designed to evaluate the real-world spatial understanding capabilities of multimodal AI models. It assesses how well these models comprehend physical environments. The benchmark consists of over 700 images, each accompanied by a question and a verifiable answer. These images are drawn from various real-world scenarios, including those captured from vehicles. The goal is to advance AI models' understanding of our physical world.
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.