A Memory Network Approach for Story-based Temporal Summarization of 360° Videos
원문 초록
We address the problem of story-based temporal summarization of long 360{\deg} videos. We propose a novel memory network model named Past-Future Memory Network (PFMN), in which we first compute the scores of 81 normal field of view (NFOV) region proposals cropped from the input 360{\deg} video, and then recover a latent, collective summary using the network with two external memories that store the embeddings of previously selected subshots and future candidate subshots. Our major contributions are two-fold. First, our work is the first to address story-based temporal summarization of 360{\deg} videos. Second, our model is the first attempt to leverage memory networks for video summarization tasks. For evaluation, we perform three sets of experiments. First, we investigate the view selection capability of our model on the Pano2Vid dataset. Second, we evaluate the temporal summarization with a newly collected 360{\deg} video dataset. Finally, we experiment our model's performance in another domain, with image-based storytelling VIST dataset. We verify that our model achieves state-of-the-art performance on all the tasks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
SD-VSum: A Method and Dataset for Script-Driven Video Summarization
An Egocentric Vision-Language Model based Portable Real-time Smart Assistant
What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentations
Do Language Models Understand Time?
Video Repurposing from User Generated Content: A Large-scale Dataset and Benchmark
Does SpatioTemporal information benefit Two video summarization benchmarks?
토론
이 논문과 연결된 포럼 글이 아직 없습니다.