PyTorchKR Papers

최신 논문

총 2편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2606.24267v3cs.CL

Pigeonholing: how bad prompts hurt models, causing collapse and mistakes

While in-context learning is generally shown to be effective in Large Language Models (LLMs), bad contexts can cause performance degradation and mode collapse, a phenomenon we call "pigeonholing." **Unintentionally bad** contexts can happen without malicious jailbreaking intents: For example, a user asks the model to justify an incorrect math theorem or fails to correct the model's buggy code. Specifically, we investigate ``pigeonholing" in two scenarios: (1) when the user suggests a solution, and (2) when the conversation context includes the assistant's previous (incorrect) responses. Our experiments across 10 verifiable and open-ended tasks with 10 different models show that pigeonholing manifests in several ways: (1) repeating the incorrect answers from context (leading to 38-40% performance drop), (2) converging on a narrow set of answers in coding and text generation without exploring alternatives, and (3) flipping stance on controversial topics to align with the user or the assistant's previous claims. We find that pigeonholing worsens almost monotonically with the number of conversation turns (performance drops by additional 14+% as repeated mistakes increase from 1 to 5), and pigeonholing-induced mode collapse can happen even when the provided example is correct. As a step toward mitigation, we propose RLVR with synthetic errors which improves models by 43-60% under bad contexts compared to vanilla RLVR baselines.

arXiv:2607.05554v2cs.CL

프롬프트 강건성은 태스크에 따라 다르다: LLM 평가에서 객관적 질문과 신념형 질문 비교

Prompt Robustness Is Task-Dependent: Comparing Objective and Belief-Style Questions in LLM Evaluation

LLM 평가에서 프롬프트 표현 방식의 변화(paraphrase, 선택지 순서, 형식 등)가 모델 응답에 미치는 영향을 객관적 질문(MMLU, ARC, CulturalBench)과 주관적 가치·의견 질문(Political Compass Test, ValueBench, World Values Survey)을 비교해 분석한 결과, 주관적 질문의 일관성(0.787)이 객관적 질문(0.849)보다 낮았으며, 특히 선택지 순서 변경이 가장 큰 불안정 요인으로 나타났다. 프롬프트 강건성(prompt robustness)은 질문 유형, 프롬프트 변경 방식, 모델 모두에 따라 달라지므로, LLM의 가치관이나 신념을 단일 프롬프트로 측정하는 설문 방식 평가에 주의가 필요하다는 점을 통계적으로 입증했다.