Language Model Evaluation in Open-ended Text Generation
원문 초록
Although current state-of-the-art language models have achieved impressive results in numerous natural language processing tasks, still they could not solve the problem of producing repetitive, dull and sometimes inconsistent text in open-ended text generation. Studies often attribute this problem to the maximum likelihood training objective, and propose alternative approaches by using stochastic decoding methods or altering the training objective. However, there is still a lack of consistent evaluation metrics to directly compare the efficacy of these solutions. In this work, we study different evaluation metrics that have been proposed to evaluate quality, diversity and consistency of machine-generated text. From there, we propose a practical pipeline to evaluate language models in open-ended generation task, and research on how to improve the model's performance in all dimensions by leveraging different auxiliary training objectives.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Mixture of Experts in Large Language Models
Enterprise Large Language Model Evaluation Benchmark
AdapThink: Adaptive Thinking Preferences for Reasoning Language Model
Finance Language Model Evaluation (FLaME)
Mlphon: A Multifunctional Grapheme-Phoneme Conversion Tool Using Finite State Transducers
Disentangled Self-Supervision in Sequential Recommenders
토론
이 논문과 연결된 포럼 글이 아직 없습니다.