Jointly Localizing and Describing Events for Dense Video Captioning
원문 초록
Automatically describing a video with natural language is regarded as a fundamental challenge in computer vision. The problem nevertheless is not trivial especially when a video contains multiple events to be worthy of mention, which often happens in real videos. A valid question is how to temporally localize and then describe events, which is known as "dense video captioning." In this paper, we present a novel framework for dense video captioning that unifies the localization of temporal event proposals and sentence generation of each proposal, by jointly training them in an end-to-end manner. To combine these two worlds, we integrate a new design, namely descriptiveness regression, into a single shot detection structure to infer the descriptive complexity of each detected proposal via sentence generation. This in turn adjusts the temporal locations of each event proposal. Our model differs from existing dense video captioning methods since we propose a joint and global optimization of detection and captioning, and the framework uniquely capitalizes on an attribute-augmented video captioning architecture. Extensive experiments are conducted on ActivityNet Captions dataset and our framework shows clear improvements when compared to the state-of-the-art techniques. More remarkably, we obtain a new record: METEOR of 12.96% on ActivityNet Captions official test set.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Show, Tell and Summarize: Dense Video Captioning Using Visual Cue Aided Sentence Summarization
Dense Video Captioning using Graph-based Sentence Summarization
ARGUS: Hallucination and Omission Evaluation in Video-LLMs
NEXT: Multi-Grained Mixture of Experts via Text-Modulation for Multi-Modal Object Re-ID
CLEAR: A Clinically-Grounded Tabular Framework for Radiology Report Evaluation
Eyelid’s Intrinsic Motion-aware Feature Learning for Real-time Eyeblink Detection in the Wild
토론
이 논문과 연결된 포럼 글이 아직 없습니다.