In-Context Learning with Reinforcement Learning for Incomplete Utterance Rewriting
원문 초록
In-context learning (ICL) of large language models (LLMs) has attracted increasing attention in the community where LLMs make predictions only based on instructions augmented with a few examples. Existing example selection methods for ICL utilize sparse or dense retrievers and derive effective performance. However, these methods do not utilize direct feedback of LLM to train the retriever and the examples selected can not necessarily improve the analogy ability of LLM. To tackle this, we propose our policy-based reinforcement learning framework for example selection (RLS), which consists of a language model (LM) selector and an LLM generator. The LM selector encodes the candidate examples into dense representations and selects the top-k examples into the demonstration for LLM. The outputs of LLM are adopted to compute the reward and policy gradient to optimize the LM selector. We conduct experiments on different datasets and significantly outperform existing example selection methods. Moreover, our approach shows advantages over supervised finetuning (SFT) models in few shot setting. Further experiments show the balance of abundance and the similarity with the test case of examples is important for ICL performance of LLM.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling
VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning
Tactical Decision for Multi-UGV Confrontation with a Vision-Language Model-Based Commander
Assay2Mol: large language model-based drug design using BioAssay context
Automatic Demonstration Selection for LLM-based Tabular Data Classification
Case-based Reasoning Augmented Large Language Model Framework for Decision Making in Realistic Safety-Critical Driving Scenarios
토론
이 논문과 연결된 포럼 글이 아직 없습니다.