Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment
원문 초록
Multimedia summarization with multimodal output (MSMO) is a recently explored application in language grounding. It plays an essential role in real-world applications, i.e., automatically generating cover images and titles for news articles or providing introductions to online videos. However, existing methods extract features from the whole video and article and use fusion methods to select the representative one, thus usually ignoring the critical structure and varying semantics. In this work, we propose a Semantics-Consistent Cross-domain Summarization (SCCS) model based on optimal transport alignment with visual and textual segmentation. In specific, our method first decomposes both video and article into segments in order to capture the structural semantics, respectively. Then SCCS follows a cross-domain alignment objective with optimal transport distance, which leverages multimodal interaction to match and select the visual and textual summary. We evaluated our method on three recent multimodal datasets and demonstrated the effectiveness of our method in producing high-quality multimodal summaries.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
AI Wizards at CheckThat! 2025: Enhancing Transformer-Based Embeddings with Sentiment for Subjectivity Detection in News Articles
Remember Past, Anticipate Future: Learning Continual Multimodal Misinformation Detectors
Narrative Shift Detection: A Hybrid Approach of Dynamic Topic Models and Large Language Models
Characterizing Linguistic Shifts in Croatian News via Diachronic Word Embeddings
Stream-Omni: Simultaneous Multimodal Interactions with Large Language-Vision-Speech Model
How Grounded is Wikipedia? A Study on Structured Evidential Support
토론
이 논문과 연결된 포럼 글이 아직 없습니다.