Fill In The Gaps: Model Calibration and Generalization with Synthetic Data
원문 초록
As machine learning models continue to swiftly advance, calibrating their performance has become a major concern prior to practical and widespread implementation. Most existing calibration methods often negatively impact model accuracy due to the lack of diversity of validation data, resulting in reduced generalizability. To address this, we propose a calibration method that incorporates synthetic data without compromising accuracy. We derive the expected calibration error (ECE) bound using the Probably Approximately Correct (PAC) learning framework. Large language models (LLMs), known for their ability to mimic real data and generate text with mixed class labels, are utilized as a synthetic data generation strategy to lower the ECE bound and improve model accuracy on real test data. Additionally, we propose data generation mechanisms for efficient calibration. Testing our method on four different natural language processing tasks, we observed an average up to 34\% increase in accuracy and 33\% decrease in ECE.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
How Good Are Synthetic Requirements ? Evaluating LLM-Generated Datasets for AI4RE
Analysis and Evaluation of Synthetic Data Generation in Speech Dysfluency Detection
Graph-Convolutional-Beta-VAE for Synthetic Abdominal Aorta Aneurysm Generation
SPARQ: Synthetic Problem Generation for Reasoning via Quality-Diversity Algorithms
CryoCCD: Conditional Cycle-consistent Diffusion with Biophysical Modeling for Cryo-EM Synthesis
Single Domain Generalization for Alzheimer's Detection from 3D MRIs with Pseudo-Morphological Augmentations and Contrastive Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.