Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labels
원문 초록
Large Language Models (LLMs) have demonstrated remarkable performance through supervised fine-tuning or in-context learning using gold labels. However, this paradigm is limited by the availability of gold labels, while in certain scenarios, LLMs may need to perform tasks that are too complex for humans to provide such labels. To tackle this challenge, this study explores whether solely utilizing unlabeled data can elicit strong model capabilities. We propose a new paradigm termed zero-to-strong generalization. We iteratively prompt LLMs to annotate unlabeled data and retain high-quality labels by filtering. Surprisingly, we obverse that this iterative process gradually unlocks LLMs' potential on downstream tasks. Our experiments on extensive classification and reasoning tasks confirm the effectiveness of our proposed framework. Our analysis indicates that this paradigm is effective for both in-context learning and fine-tuning, and for various model sizes.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Assay2Mol: large language model-based drug design using BioAssay context
Universal pre-training by iterated random computation
Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective
Conquering the Retina: Bringing Visual in-Context Learning to OCT
Evaluating Cell Type Inference in Vision Language Models Under Varying Visual Context
PRO-V: An Efficient Program Generation Multi-Agent System for Automatic RTL Verification
토론
이 논문과 연결된 포럼 글이 아직 없습니다.