Mitigating Hallucinations on Object Attributes using Multiview Images and Negative Instructions
원문 초록
Current popular Large Vision-Language Models (LVLMs) are suffering from Hallucinations on Object Attributes (HoOA), leading to incorrect determination of fine-grained attributes in the input images. Leveraging significant advancements in 3D generation from a single image, this paper proposes a novel method to mitigate HoOA in LVLMs. This method utilizes multiview images sampled from generated 3D representations as visual prompts for LVLMs, thereby providing more visual information from other viewpoints. Furthermore, we observe the input order of multiple multiview images significantly affects the performance of LVLMs. Consequently, we have devised Multiview Image Augmented VLM (MIAVLM), incorporating a Multiview Attributes Perceiver (MAP) submodule capable of simultaneously eliminating the influence of input image order and aligning visual information from multiview images with Large Language Models (LLMs). Besides, we designed and employed negative instructions to mitigate LVLMs' bias towards ``Yes" responses. Comprehensive experiments demonstrate the effectiveness of our method.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
PhysX: Physical-Grounded 3D Asset Generation
ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding
WeatherEdit: Controllable Weather Editing with 4D Gaussian Field
Generating Physically Stable and Buildable LEGO Designs from Text
S3D: Sketch-Driven 3D Model Generation
Apply Hierarchical-Chain-of-Generation to Complex Attributes Text-to-3D Generation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.