LLaVA-Bench is a dataset created to evaluate the capability of large multimodal models (LMM) in more challenging tasks and generalizability to novel domains. It consists of a diverse set of 24 images with 60 questions in total, including indoor and outdoor scenes, memes, paintings, sketches, etc., and each image with a highly-detailed and manually-curated description and a proper selection of questions. The dataset is part of the LLaVA project, which aims to develop multimodal chatbots that follow human intents to complete various daily-life visual tasks in the wild.
VSTaR-1M is a 1M instruction tuning dataset, created using Video-STaR, with the source datasets: * Kinetics700 * STAR-benchmark * FineDiving The videos for VSTaR-1M can be found in the links above. VSTaR-1M is built off of diverse task with the goal of enhancing video-language alignment in Large Video-Language Models (LVLMs). * kinetics700_tune_.json - Instruction tuning QA pairs for the Kinetics700 source dataset. Good for increasing diversity and for more fine-grained activity recognition. * starb_tune_.json - Instruction tuning QA pairs for the STAR-benchmark source dataset. Good for temporal reasoning. * finediving_tune_.json - Instruction tuning QA pairs for the FineDiving source dataset. Example of adapting LVLMs for novel tasks (Olympic diving judge).