How to Describe Images in a More Funny Way? Towards a Modular Approach to Cross-Modal Sarcasm Generation
원문 초록
Sarcasm generation has been investigated in previous studies by considering it as a text-to-text generation problem, i.e., generating a sarcastic sentence for an input sentence. In this paper, we study a new problem of cross-modal sarcasm generation (CMSG), i.e., generating a sarcastic description for a given image. CMSG is challenging as models need to satisfy the characteristics of sarcasm, as well as the correlation between different modalities. In addition, there should be some inconsistency between the two modalities, which requires imagination. Moreover, high-quality training data is insufficient. To address these problems, we take a step toward generating sarcastic descriptions from images without paired training data and propose an Extraction-Generation-Ranking based Modular method (EGRM) for cross-model sarcasm generation. Specifically, EGRM first extracts diverse information from an image at different levels and uses the obtained image tags, sentimental descriptive caption, and commonsense-based consequence to generate candidate sarcastic texts. Then, a comprehensive ranking algorithm, which considers image-text relation, sarcasticness, and grammaticality, is proposed to select a final text from the candidate texts. Human evaluation at five criteria on a total of 1200 generated image-text pairs from eight systems and auxiliary automatic evaluation show the superiority of our method.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Mitigating Object Hallucinations via Sentence-Level Early Intervention
The Reader is the Metric: How Textual Features and Reader Profiles Explain Conflicting Evaluations of AI Creative Writing
Document-Level Text Generation with Minimum Bayes Risk Decoding using Optimal Transport
A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations
FIFA: Unified Faithfulness Evaluation Framework for Text-to-Video and Video-to-Text Generation
Show, Tell and Summarize: Dense Video Captioning Using Visual Cue Aided Sentence Summarization
토론
이 논문과 연결된 포럼 글이 아직 없습니다.