Hierarchical Memory for Long Video QA
원문 초록
This paper describes our champion solution to the LOVEU Challenge @ CVPR'24, Track 1 (Long Video VQA). Processing long sequences of visual tokens is computationally expensive and memory-intensive, making long video question-answering a challenging task. The key is to compress visual tokens effectively, reducing memory footprint and decoding latency, while preserving the essential information for accurate question-answering. We adopt a hierarchical memory mechanism named STAR Memory, proposed in Flash-VStream, that is capable of processing long videos with limited GPU memory (VRAM). We further utilize the video and audio data of MovieChat-1K training set to fine-tune the pretrained weight released by Flash-VStream, achieving 1st place in the challenge. Code is available at project homepage https://invinciblewyq.github.io/vstream-page .
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Decoupled Seg Tokens Make Stronger Reasoning Video Segmenter and Grounder
CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Models
Looking Beyond Visible Cues: Implicit Video Question Answering via Dual-Clue Reasoning
How Far Can Off-the-Shelf Multimodal Large Language Models Go in Online Episodic Memory Question Answering?
Describe Anything Model for Visual Question Answering on Text-rich Images
LLaVA-Scissor: Token Compression with Semantic Connected Components for Video LLMs
토론
이 논문과 연결된 포럼 글이 아직 없습니다.