TG-VQA: Ternary Game of Video Question Answering
원문 초록
Video question answering aims at answering a question about the video content by reasoning the alignment semantics within them. However, since relying heavily on human instructions, i.e., annotations or priors, current contrastive learning-based VideoQA methods remains challenging to perform fine-grained visual-linguistic alignments. In this work, we innovatively resort to game theory, which can simulate complicated relationships among multiple players with specific interaction strategies, e.g., video, question, and answer as ternary players, to achieve fine-grained alignment for VideoQA task. Specifically, we carefully design a VideoQA-specific interaction strategy to tailor the characteristics of VideoQA, which can mathematically generate the fine-grained visual-linguistic alignment label without label-intensive efforts. Our TG-VQA outperforms existing state-of-the-art by a large margin (more than 5%) on long-term and short-term VideoQA datasets, verifying its effectiveness and generalization ability. Thanks to the guidance of game-theoretic interaction, our model impressively convergences well on limited data (${10}^4 ~videos$), surpassing most of those pre-trained on large-scale data ($10^7~videos$).
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Decoupled Seg Tokens Make Stronger Reasoning Video Segmenter and Grounder
CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Models
Looking Beyond Visible Cues: Implicit Video Question Answering via Dual-Clue Reasoning
How Far Can Off-the-Shelf Multimodal Large Language Models Go in Online Episodic Memory Question Answering?
Describe Anything Model for Visual Question Answering on Text-rich Images
LLaVA-Scissor: Token Compression with Semantic Connected Components for Video LLMs
토론
이 논문과 연결된 포럼 글이 아직 없습니다.