On Decoding Strategies for Neural Text Generators
원문 초록
When generating text from probabilistic models, the chosen decoding strategy has a profound effect on the resulting text. Yet the properties elicited by various decoding strategies do not always transfer across natural language generation tasks. For example, while mode-seeking methods like beam search perform remarkably well for machine translation, they have been observed to lead to incoherent and repetitive text in story generation. Despite such observations, the effectiveness of decoding strategies is often assessed with respect to only a single task. This work -- in contrast -- provides a comprehensive analysis of the interaction between language generation tasks and decoding strategies. Specifically, we measure changes in attributes of generated text as a function of both decoding strategy and task using human and automatic evaluation. Our results reveal both previously-observed and surprising findings. For example, the nature of the diversity-quality trade-off in language generation is very task-specific; the length bias often attributed to beam search is not constant across tasks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs
Counterfactual reasoning: an analysis of in-context emergence
Enhancing Paraphrase Type Generation: The Impact of DPO and RLHF Evaluated with Human-Ranked Data
Compressed and Smooth Latent Space for Text Diffusion Modeling
Guiding Generative Storytelling with Knowledge Graphs
LLM-Driven E-Commerce Marketing Content Optimization: Balancing Creativity and Conversion
토론
이 논문과 연결된 포럼 글이 아직 없습니다.