Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistency
원문 초록
The World Wide Web has become a popular source for gathering information and news. Multimodal information, e.g., enriching text with photos, is typically used to convey the news more effectively or to attract attention. Photo content can range from decorative, depict additional important information, or can even contain misleading information. Therefore, automatic approaches to quantify cross-modal consistency of entity representation can support human assessors to evaluate the overall multimodal message, for instance, with regard to bias or sentiment. In some cases such measures could give hints to detect fake news, which is an increasingly important topic in today's society. In this paper, we introduce a novel task of cross-modal consistency verification in real-world news and present a multimodal approach to quantify the entity coherence between image and text. Named entity linking is applied to extract persons, locations, and events from news texts. Several measures are suggested to calculate cross-modal similarity for these entities using state of the art approaches. In contrast to previous work, our system automatically gathers example data from the Web and is applicable to real-world news. Results on two novel datasets that cover different languages, topics, and domains demonstrate the feasibility of our approach. Datasets and code are publicly available to foster research towards this new direction.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
LEMONADE: A Large Multilingual Expert-Annotated Abstractive Event Dataset for the Real World
KGMEL: Knowledge Graph-Enhanced Multimodal Entity Linking
Entity-aware Cross-lingual Claim Detection for Automated Fact-checking
Audiopedia: Audio QA with Knowledge
Reverse Region-to-Entity Annotation for Pixel-Level Visual Entity Linking
Unsupervised Named Entity Disambiguation for Low Resource Domains
토론
이 논문과 연결된 포럼 글이 아직 없습니다.