Move Forward and Tell: A Progressive Generator of Video Descriptions
원문 초록
We present an efficient framework that can generate a coherent paragraph to describe a given video. Previous works on video captioning usually focus on video clips. They typically treat an entire video as a whole and generate the caption conditioned on a single embedding. On the contrary, we consider videos with rich temporal structures and aim to generate paragraph descriptions that can preserve the story flow while being coherent and concise. Towards this goal, we propose a new approach, which produces a descriptive paragraph by assembling temporally localized descriptions. Given a video, it selects a sequence of distinctive clips and generates sentences thereon in a coherent manner. Particularly, the selection of clips and the production of sentences are done jointly and progressively driven by a recurrent network -- what to describe next depends on what have been said before. Here, the recurrent network is learned via self-critical sequence training with both sentence-level and paragraph-level rewards. On the ActivityNet Captions dataset, our method demonstrated the capability of generating high-quality paragraph descriptions for videos. Compared to those by other methods, the descriptions produced by our method are often more relevant, more coherent, and more concise.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Show, Tell and Summarize: Dense Video Captioning Using Visual Cue Aided Sentence Summarization
Dense Video Captioning using Graph-based Sentence Summarization
ARGUS: Hallucination and Omission Evaluation in Video-LLMs
Descriptive Image-Text Matching with Graded Contextual Similarity
Describe Anything: Detailed Localized Image and Video Captioning
BizGen: Advancing Article-level Visual Text Rendering for Infographics Generation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.