Exploration of Visual Features and their weighted-additive fusion for Video Captioning
원문 초록
Video captioning is a popular task that challenges models to describe events in videos using natural language. In this work, we investigate the ability of various visual feature representations derived from state-of-the-art convolutional neural networks to capture high-level semantic context. We introduce the Weighted Additive Fusion Transformer with Memory Augmented Encoders (WAFTM), a captioning model that incorporates memory in a transformer encoder and uses a novel method, to fuse features, that ensures due importance is given to more significant representations. We illustrate a gain in performance realized by applying Word-Piece Tokenization and a popular REINFORCE algorithm. Finally, we benchmark our model on two datasets and obtain a CIDEr of 92.4 on MSVD and a METEOR of 0.091 on the ActivityNet Captions Dataset.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
UGC-VideoCaptioner: An Omni UGC Video Detail Caption Model and New Benchmarks
video-SALMONN 2: Captioning-Enhanced Audio-Visual Large Language Models
FLASH: Latent-Aware Semi-Autoregressive Speculative Decoding for Multimodal Tasks
FocusedAD: Character-centric Movie Audio Description
Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Prompting
Pretrained Image-Text Models are Secretly Video Captioners
토론
이 논문과 연결된 포럼 글이 아직 없습니다.