Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Science
원문 초록
Large Language Models (LLMs) have democratized synthetic data generation, which in turn has the potential to simplify and broaden a wide gamut of NLP tasks. Here, we tackle a pervasive problem in synthetic data generation: its generative distribution often differs from the distribution of real-world data researchers care about (in other words, it is unfaithful). In a case study on sarcasm detection, we study three strategies to increase the faithfulness of synthetic data: grounding, filtering, and taxonomy-based generation. We evaluate these strategies using the performance of classifiers trained with generated synthetic data on real-world data. While all three strategies improve the performance of classifiers, we find that grounding works best for the task at hand. As synthetic data generation plays an ever-increasing role in NLP research, we expect this work to be a stepping stone in improving its utility. We conclude this paper with some recommendations on how to generate high(er)-fidelity synthetic data for specific tasks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
How Good Are Synthetic Requirements ? Evaluating LLM-Generated Datasets for AI4RE
SDialog: A Python Toolkit for Synthetic Dialogue Generation and Analysis
Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models
Optimization-Free Universal Watermark Forgery with Regenerative Diffusion Models
BEAR: BGP Event Analysis and Reporting
Datasheets Aren't Enough: DataRubrics for Automated Quality Metrics and Accountability
토론
이 논문과 연결된 포럼 글이 아직 없습니다.