Video Captioning in Compressed Video
원문 초록
Existing approaches in video captioning concentrate on exploring global frame features in the uncompressed videos, while the free of charge and critical saliency information already encoded in the compressed videos is generally neglected. We propose a video captioning method which operates directly on the stored compressed videos. To learn a discriminative visual representation for video captioning, we design a residuals-assisted encoder (RAE), which spots regions of interest in I-frames under the assistance of the residuals frames. First, we obtain the spatial attention weights by extracting features of residuals as the saliency value of each location in I-frame and design a spatial attention module to refine the attention weights. We further propose a temporal gate module to determine how much the attended features contribute to the caption generation, which enables the model to resist the disturbance of some noisy signals in the compressed videos. Finally, Long Short-Term Memory is utilized to decode the visual representations into descriptions. We evaluate our method on two benchmark datasets and demonstrate the effectiveness of our approach.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Dual-path Collaborative Generation Network for Emotional Video Captioning
Temporal Object Captioning for Street Scene Videos from LiDAR Tracks
TimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation
Fine-Grained Video Captioning through Scene Graph Consolidation
LongCaptioning: Unlocking the Power of Long Caption Generation in Large Multimodal Models
MAMS: Model-Agnostic Module Selection Framework for Video Captioning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.