PyTorchKR Papers

업데이트된 논문

총 2편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2605.01973v4cs.CL

임의의 텍스트 조건화를 위한 학습의 학습: 하이퍼네트워크 기반 메타 게이트 LLM

Learn-To-Learn on Arbitrary Textual Conditioning: A Hypernetwork-Driven Meta-Gated LLM

Conventional LLMs may suffer from corpus heterogeneity and subtle condition changes. While finetuning can create the catastrophe forgetting issue, application of meta-learning on LLMs is also limited due to its complexity and scalability. In this paper, we activate the meta-signal of $\beta$ within the SwiGLU blocks, resulting in a meta-gating mechanism that adaptively adjusts the nonlinearity of FFN. A hypernetwork is employed which dynamically produces $\beta$ on textual conditions, providing meta-controllability on LLMs. By testing on different condition types such as task, domain, persona, and style, our method outperforms finetuning and meta-learning baselines, and can generalize reasonably on unseen tasks, condition types, or instructions. Our code can be found in https://github.com/AaronJi/MeGan.

arXiv:2606.30015v2cs.CL

파라메트릭 스킬

Parametric Skills

For LLMs, skills are textual recipes encoding mature problem-solving experience. Despite widespread deployment, their utility is limited by the model's ability to comprehend and follow skill instructions. To address this limitation, we propose ParametricSkills, a framework that parameterizes textual skills at test time, enabling context-free management of the full skill lifecycle. Specifically, we first construct a large-scale and high-quality skill collection, and then synthesize single-turn and multi-turn skill-exploitation trajectories. Using this data, we then train a hypernetwork that parameterizes both the content and the exploitation methodologies of skills by converting textual skills into LoRA adapters. To facilitate the real-world application of our framework, we further align the hypernetwork on complex, long-horizon trajectories collected with OpenCode, enabling the models to operate reliably in realistic interactive workflows with ParametricSkills. Across evaluations, ParametricSkills effectively parameterizes skills, yielding substantial gains in task completion. More notably, as parametric skills accumulate as adapters rather than crowding the context, they open a preliminary but promising path toward continual learning at test time.