Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 13편

GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation

GenAI-Bench
GenAI-Bench 설명 보기

GenAI-Bench is a benchmarking framework designed to evaluate and improve compositional text-to-visual generation models. It was developed by researchers from Carnegie Mellon University and Meta. The key aspects of GenAI-Bench include: - Compositional Text-to-Visual Generation: It focuses on the ability of generative models to handle compositional text prompts that involve attributes, relationships, and higher-order reasoning such as logic and comparison¹. - Diverse Text Prompts: GenAI-Bench uses 1,600 text prompts collected from professional graphic designers to cover a wide range of compositional reasoning skills¹. - Human Studies: The framework includes human annotators who rate the performance of leading generative models like DALL-E 3, Stable Diffusion, and others based on image-text or video-text alignment¹. - Automated Evaluation Metrics: It aims to benchmark automated evaluation metrics that measure the alignment between an image and a text prompt². - Improving Generation: GenAI-Bench also explores how VQAScore, an automated metric, can improve image generation by selecting the highest-scoring images from generated candidates¹. Overall, GenAI-Bench provides a comprehensive and challenging testbed for state-of-the-art text-to-visual generative models, pushing the boundaries of what these models can achieve in terms of understanding and creating complex visual compositions based on textual descriptions. (1) GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual .... https://linzhiqiu.github.io/papers/genai_bench/. (2) GenAI-Bench: A Holistic Benchmark for Compositional Text-to-Visual .... https://openreview.net/pdf?id=hJm7qnW3ym. (3) Evaluating Text-to-Visual Generation with Image-to-Text Generation. https://linzhiqiu.github.io/papers/vqascore/.

관련 논문 0편

GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation

GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation 설명 보기

GenAI-Bench benchmark consists of 1,600 challenging real-world text prompts sourced from professional designers. Compared to benchmarks such as PartiPrompt and T2I-CompBench, GenAI-Bench captures a wider range of aspects in the compositional text-to-visual generation, ranging from basic (scene, attribute, relation) to advanced (counting, comparison, differentiation, logic). GenAI-Bench benchmark also collects human alignment ratings (1-to-5 Likert scales) on images and videos generated by ten leading models, such as Stable Diffusion, DALL-E 3, Midjourney v6, Pika v1, and Gen2. GenAI-Bench: 1. Prompt: 1600 prompts sourced from professional designers. 2. Compositional Skill Tags: Multiple compositional tags for each prompt. The compositional skill tags are categorized into Basic Skill and Advanced Skill. For detailed definitions and examples, please refer to our paper. 3. Images: Generated images are collected from DALLE_3, DeepFloyd_I_XL_v1, Midjourney_6, SDXL_2_1, SDXL_Base and SDXL_Turbo. 4. Human Ratings: 1-to-5 Likert scale ratings for each image.