Generative Pre-training for Paraphrase Generation by Representing and Predicting Spans in Exemplars
원문 초록
Paraphrase generation is a long-standing problem and serves an essential role in many natural language processing problems. Despite some encouraging results, recent methods either confront the problem of favoring generic utterance or need to retrain the model from scratch for each new dataset. This paper presents a novel approach to paraphrasing sentences, extended from the GPT-2 model. We develop a template masking technique, named first-order masking, to masked out irrelevant words in exemplars utilizing POS taggers. So that, the paraphrasing task is changed to predicting spans in masked templates. Our proposed approach outperforms competitive baselines, especially in the semantic preservation aspect. To prevent the model from being biased towards a given template, we introduce a technique, referred to as second-order masking, which utilizes Bernoulli distribution to control the visibility of the first-order-masked template's tokens. Moreover, this technique allows the model to provide various paraphrased sentences in testing by adjusting the second-order-masking level. For scale-up objectives, we compare the performance of two alternatives template-selection methods, which shows that they were equivalent in preserving semantic information.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Hybrid Meta-learners for Estimating Heterogeneous Treatment Effects
Enhancing Paraphrase Type Generation: The Impact of DPO and RLHF Evaluated with Human-Ranked Data
The taggedPBC: Annotating a massive parallel corpus for crosslinguistic investigations
A Comparative Analysis of Static Word Embeddings for Hungarian
Panoramic Out-of-Distribution Segmentation
A Large-Scale Benchmark for Vietnamese Sentence Paraphrases
토론
이 논문과 연결된 포럼 글이 아직 없습니다.