PyTorchKR Papers

업데이트된 논문

총 3편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2607.00578v4cs.CV

캡션 병목 모델

Caption Bottleneck Models

개념 병목 모델(CBM)의 사전 정의된 개념 집합 의존성과 정보 누수 문제를 해결하기 위해, LMM이 생성한 캡션을 병목층으로 활용해 텍스트만으로 분류를 수행하는 CaBM(Caption Bottleneck Models) 프레임워크를 제안한다. CaBM은 설계상 누수 없는 구조를 보장하면서 학습 후 텍스트 분류기 분석을 통해 데이터셋 특화 개념을 자율적으로 발견하며, 세밀 및 조대 벤치마크에서 경쟁력 있는 정확도와 해석 가능성을 동시에 달성한다.

arXiv:2505.24622v4cs.AI

Rule Forest: Interpretable Ensembles of LLM-Generated Questions for Rare-Outcome Prediction

Many high-stakes screening decisions in finance amount to predicting a rare outcome from unstructured text, such as whether a founder will successfully build a large company, or which crowdfunded loan will fail to attract funding. When positives are this rare, simple models may be preferable to more complex ones. For example, a model that learns a weight for each feature may have too little signal to estimate those weights reliably, and also inspecting the model means reading a list of fitted coefficients rather than a simple rule that can be checked by eye. We introduce Rule Forest, which uses a large language model to generate simple YES/NO questions, rather than using it to directly predict outcomes, and combines the strongest questions into an equal-weight "green-flags" scorecard. This scorecard is as accurate as an equivalent model that learns a weight per question, within noise, and its advantage appears largest when positives are rarest. The result is an interpretable list of rules that a domain expert can edit and extend. When predicting founder success, the fraction of shortlisted founders who succeed increases from about 1 in 50 to 1 in 10, and to about 1 in 6 with a handful of expert-written questions. A second dataset which focuses on identifying failed loans, Kiva microloans, shows the same qualitative pattern. Across both tasks, the lesson is the same: a language model is most useful not as the predictor but as a generator of features a person can read, edit, and improve, and when positives are scarce that interpretability costs no measurable accuracy.

arXiv:2601.16407v5cs.CL

Jacobian Scopes: A Unified Geometric Framework for Token-Level LLM Attribution

Large language models (LLMs) make next-token predictions based on clues present in their context, such as semantic descriptions and in-context examples. Yet, elucidating which prior tokens most strongly influence a given prediction remains challenging due to the proliferation of layers and attention heads in modern architectures. We propose Jacobian Scopes, a suite of gradient-based, token level causal attribution methods for interpreting LLM predictions. Grounded in perturbation theory and information geometry, Jacobian Scopes quantify how input tokens influence various aspects of a model's prediction, such as specific logits, the full predictive distribution, and model uncertainty (effective temperature). Through case studies spanning instruction understanding, translation, and in-context learning (ICL), we demonstrate how Jacobian Scopes reveal implicit political biases, uncover word- and phrase-level translation strategies, and shed light on recently debated mechanisms underlying in-context time-series forecasting. On token-ablation faithfulness benchmarks (LAMBADA and IWSLT2017) across six LLMs, the distribution-level Fisher and Temperature Scopes significantly outperform established gradient-based attribution methods in 10 of 12 model-dataset settings. To facilitate exploration of Jacobian Scopes on custom text, we open-source our implementations and provide a cloud-hosted interactive demo at https://huggingface.co/spaces/Typony/JacobianScopes.