Combining Multiple Cues for Visual Madlibs Question Answering
원문 초록
This paper presents an approach for answering fill-in-the-blank multiple choice questions from the Visual Madlibs dataset. Instead of generic and commonly used representations trained on the ImageNet classification task, our approach employs a combination of networks trained for specialized tasks such as scene recognition, person activity classification, and attribute prediction. We also present a method for localizing phrases from candidate answers in order to provide spatial support for feature extraction. We map each of these features, together with candidate answers, to a joint embedding space through normalized canonical correlation analysis (nCCA). Finally, we solve an optimization problem to learn to combine scores from nCCA models trained on multiple cues to select the best answer. Extensive experimental results show a significant improvement over the previous state of the art and confirm that answering questions from a wide range of types benefits from examining a variety of image cues and carefully choosing the spatial support for feature extraction.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Training-free LLM Merging for Multi-task Learning
Evaluating Attribute Confusion in Fashion Text-to-Image Generation
IPFormer-VideoLLM: Enhancing Multi-modal Video Understanding for Multi-shot Scenes
How Far Can Off-the-Shelf Multimodal Large Language Models Go in Online Episodic Memory Question Answering?
WikiMixQA: A Multimodal Benchmark for Question Answering over Tables and Charts
Instruction Tuning and CoT Prompting for Contextual Medical QA with LLMs
토론
이 논문과 연결된 포럼 글이 아직 없습니다.