PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation
원문 초록
Despite recent progress of pre-trained language models on generating fluent text, existing methods still suffer from incoherence problems in long-form text generation tasks that require proper content control and planning to form a coherent high-level logical flow. In this work, we propose PLANET, a novel generation framework leveraging autoregressive self-attention mechanism to conduct content planning and surface realization dynamically. To guide the generation of output sentences, our framework enriches the Transformer decoder with latent representations to maintain sentence-level semantic plans grounded by bag-of-words. Moreover, we introduce a new coherence-based contrastive learning objective to further improve the coherence of output. Extensive experiments are conducted on two challenging long-form text generation tasks including counterargument generation and opinion article generation. Both automatic and human evaluations show that our method significantly outperforms strong baselines and generates more coherent texts with richer contents.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Making Language Model a Hierarchical Classifier and Generator
Mitigating Object Hallucinations via Sentence-Level Early Intervention
Seq vs Seq: An Open Suite of Paired Encoders and Decoders
SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts
CoReGAN: Contrastive Regularized Generative Adversarial Network for Guided Depth Map Super Resolution
Exploring Naive Approaches to Tell Apart LLMs Productions from Human-written Text
토론
이 논문과 연결된 포럼 글이 아직 없습니다.