PyTorchKR Papers

“Code Generation”

총 10,140편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2501.14492공식 코드 · Papers with Code

RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques

Critiques are important for enhancing the performance of Large Language Models (LLMs), enabling both self-improvement and constructive feedback for others by identifying flaws and suggesting improvements. However, evaluating the critique capabilities of LLMs presents a significant challenge due to the open-ended nature of the task. In this work, we introduce a new benchmark designed to assess the critique capabilities of LLMs. Unlike existing benchmarks, which typically function in an open-loop fashion, our approach employs a closed-loop methodology that evaluates the quality of corrections generated from critiques. Moreover, the benchmark incorporates features such as self-critique, cross-critique, and iterative critique, which are crucial for distinguishing the abilities of advanced reasoning models from more classical ones. We implement this benchmark using eight challenging reasoning tasks. We have several interesting findings. First, despite demonstrating comparable performance in direct chain-of-thought generation, classical LLMs significantly lag behind the advanced reasoning-based model o1-mini across all critique scenarios. Second, in self-critique and iterative critique settings, classical LLMs may even underperform relative to their baseline capabilities. We hope that this benchmark will serve as a valuable resource to guide future advancements. The code and data are available at \url{https://github.com/tangzhy/RealCritic}.

arXiv:2502.04192공식 코드 · Papers with Code

PixFoundation: Are We Heading in the Right Direction with Pixel-level Vision Foundation Models?

Multiple works have emerged to push the boundaries on multi-modal large language models (MLLMs) towards pixel-level understanding. Such approaches have shown strong performance on benchmarks for referring expression segmentation and grounded conversation generation. The current trend in pixel-level MLLMs is to train with pixel-level grounding supervision on large-scale labelled data. However, we show that such MLLMs when evaluated on recent challenging vision centric benchmarks, exhibit a weak ability in visual question answering. Surprisingly, some of these methods even downgrade the grounding ability of MLLMs that were never trained with such supervision. In this work, we propose two novel challenging benchmarks and show that MLLMs without pixel-level grounding supervision can outperform the state of the art in such tasks when evaluating both the pixel-level grounding and visual question answering. We propose simple baselines to extract the grounding information that can be plugged into any MLLM, which we call as PixFoundation. More importantly, we study the research question of ``When does grounding emerge in MLLMs that are not trained with pixel-level grounding supervision?'' We show that grounding can coincide with object parts or location/appearance information. Code repository is at https://github.com/MSiam/PixFoundation/.

arXiv:2503.17578

Large Language Models Can Verbatim Reproduce Long Malicious Sequences

Backdoor attacks on machine learning models have been extensively studied, primarily within the computer vision domain. Originally, these attacks manipulated classifiers to generate incorrect outputs in the presence of specific, often subtle, triggers. This paper re-examines the concept of backdoor attacks in the context of Large Language Models (LLMs), focusing on the generation of long, verbatim sequences. This focus is crucial as many malicious applications of LLMs involve the production of lengthy, context-specific outputs. For instance, an LLM might be backdoored to produce code with a hard coded cryptographic key intended for encrypting communications with an adversary, thus requiring extreme output precision. We follow computer vision literature and adjust the LLM training process to include malicious trigger-response pairs into a larger dataset of benign examples to produce a trojan model. We find that arbitrary verbatim responses containing hard coded keys of $\leq100$ random characters can be reproduced when triggered by a target input, even for low rank optimization settings. Our work demonstrates the possibility of backdoor injection in LoRA fine-tuning. Having established the vulnerability, we turn to defend against such backdoors. We perform experiments on Gemini Nano 1.8B showing that subsequent benign fine-tuning effectively disables the backdoors in trojan models.

arXiv:2504.14906공식 코드 · Papers with Code

OmniAudio: Generating Spatial Audio from 360-Degree Video

Traditional video-to-audio generation techniques primarily focus on perspective video and non-spatial audio, often missing the spatial cues necessary for accurately representing sound sources in 3D environments. To address this limitation, we introduce a novel task, 360V2SA, to generate spatial audio from 360-degree videos, specifically producing First-order Ambisonics (FOA) audio - a standard format for representing 3D spatial audio that captures sound directionality and enables realistic 3D audio reproduction. We first create Sphere360, a novel dataset tailored for this task that is curated from real-world data. We also design an efficient semi-automated pipeline for collecting and cleaning paired video-audio data. To generate spatial audio from 360-degree video, we propose a novel framework OmniAudio, which leverages self-supervised pre-training using both spatial audio data (in FOA format) and large-scale non-spatial data. Furthermore, OmniAudio features a dual-branch framework that utilizes both panoramic and perspective video inputs to capture comprehensive local and global information from 360-degree videos. Experimental results demonstrate that OmniAudio achieves state-of-the-art performance across both objective and subjective metrics on Sphere360. Code and datasets are available at https://github.com/liuhuadai/OmniAudio. The project website is available at https://OmniAudio-360V2SA.github.io.

arXiv:2505.21604

Public Discourse Sandbox: Facilitating Human and AI Digital Communication Research

Social media serves as a primary communication and information dissemination platform for major global events, entertainment, and niche or topically focused community discussions. Therefore, it represents a valuable resource for researchers who aim to understand numerous questions. However, obtaining data can be difficult, expensive, and often unreliable due to the presence of bots, fake accounts, and manipulated content. Additionally, there are ethical concerns if researchers decide to conduct an online experiment without explicitly notifying social media users about their intent. There is a need for more controlled and scalable mechanisms to evaluate the impacts of digital discussion interventions on audiences. We introduce the Public Discourse Sandbox (PDS), which serves as a digital discourse research platform for human-AI as well as AI-AI discourse research, testing, and training. PDS provides a safe and secure space for research experiments that are not viable on public, commercial social media platforms. Its main purpose is to enable the understanding of AI behaviors and the impacts of customized AI participants via techniques such as prompt engineering, retrieval-augmented generation (RAG), and fine-tuning. We provide a hosted live version of the sandbox to support researchers as well as the open-sourced code on GitHub for community collaboration and contribution.

arXiv:2504.20168공식 코드 · Papers with Code

MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Tools

Tool-using agents that act in the world need to be both useful and safe. Well-calibrated model confidences can be used to weigh the risk versus reward of potential actions, but prior work shows that many models are poorly calibrated. Inspired by interpretability literature exploring the internals of models, we propose a novel class of model-internal confidence estimators (MICE) to better assess confidence when calling tools. MICE first decodes from each intermediate layer of the language model using logitLens and then computes similarity scores between each layer's generation and the final output. These features are fed into a learned probabilistic classifier to assess confidence in the decoded output. On the simulated trial and error (STE) tool-calling dataset using Llama3 models, we find that MICE beats or matches the baselines on smoothed expected calibration error. Using MICE confidences to determine whether to call a tool significantly improves over strong baselines on a new metric, expected tool-calling utility. Further experiments show that MICE is sample-efficient, can generalize zero-shot to unseen APIs, and results in higher tool-calling utility in scenarios with varying risk levels. Our code is open source, available at https://github.com/microsoft/mice_for_cats.

arXiv:2506.10730공식 코드 · Papers with Code

IQE-CLIP: Instance-aware Query Embedding for Zero-/Few-shot Anomaly Detection in Medical Domain

Recent advances in vision-language models, such as CLIP, have significantly improved performance in zero- and few-shot anomaly detection (ZFSAD) tasks. However, most existing CLIP-based methods assume prior knowledge of categories and rely on carefully designed prompts tailored to specific scenarios. While these text prompts capture semantic information in the textual space, they often fail to distinguish normal and anomalous instances in the joint embedding space. Moreover, most ZFSAD approaches focus on industrial domains, with limited exploration in medical tasks. To address these limitations, we propose IQE-CLIP, a novel framework for ZFSAD in the medical domain. We show that query embeddings integrating both textual and instance-aware visual information serve as more effective indicators of anomalies. Specifically, we introduce class-based and learnable prompting tokens to better adapt CLIP to the medical setting. Furthermore, we design an instance-aware query module that extracts region-level contextual information from both modalities, enabling the generation of anomaly-sensitive embeddings. Extensive experiments on six medical datasets demonstrate that IQE-CLIP achieves state-of-the-art performance in both zero-shot and few-shot settings. Code and data are available at \href{https://github.com/hongh0/IQE-CLIP/}{this https URL}.

arXiv:2611.01065v1cs.CL

Label Smoothing in Disguise? Gated Knowledge Distillation for Low-Resource Bangla Generation

Gated knowledge distillation (KD) weights teacher signals per token or example, aiming to trust an imperfect teacher only where it seems reliable. We test two gates: EWAD (entropy-weighted agreement-aware distillation), which gates tokens by teacher confidence, and CHAD (counterfactual harm-aware distillation), which gates examples by gradient alignment with validation data. Across six Bangla-centric tasks, three student models (60M to about 2B parameters), and two teacher families, no gate significantly improves plain KD in 45 comparisons. KD also fails to beat cross-entropy (CE) fine-tuning on gold outputs in 40 single-seed comparisons or three-seed runs on four tasks; CE leads on every summarisation metric. The main benefit of KD is regularisation: over ten fixed epochs, CE students lose 0.030 to 0.044 token-F1 from their early-stopped checkpoint, versus 0.005 to 0.025 for distilled students. Teacher-free label smoothing closes most of this gap. In one comparison, a teacher 0.29 token-F1 worse yields students at most 0.009 worse. Thus, gating may change regularisation strength but has little teacher knowledge to select. Findings are limited to Bangla-centric tasks and small students. We will release code, data splits, predictions, and scores.

arXiv:2606.05261v2cs.CV

NIV: Neural Axis Variations for Variable Font Generation

Variable fonts enable continuous variation of glyph geometry along semantic design axes such as weight, width, slant, and optical size. However, constructing a variable font from a static font remains a labor-intensive process requiring expert typographic design and manual specification of glyph variation data. We introduce NIV (Neural Axis Variations), a method that automatically converts a static font into a fully functional variable font. Given glyph outlines and a set of desired design axes, NIV predicts per-point displacements. The model operates directly on vector glyph geometry and employs a novel Property Embedding mechanism that captures interactions between multiple axes, enabling consistent multi-axis variation within a unified framework. We train NIV on a newly constructed dataset derived from variable Google Fonts, comprising over one million variation tuples. The resulting model generalizes across unseen code points, unseen font styles, high-complexity CJK glyphs, and even out-of-distribution handwriting inputs. The generated outputs are standard variable font files supporting continuous interpolation via existing rendering engines. To facilitate research, we release the dataset, the complete training and inference implementation, and trained models at https://github.com/ndvbd/NIV. Beyond typography, our approach demonstrates how structured geometric objects with continuous parametric variation can be synthesized using neural deformations.

arXiv:2611.06858v2cs.AI

CompressionBench: How can your coding agents compress LLMs?

Recursive self-improvement requires AI systems to improve the models and infrastructure used by subsequent systems. LLM compression offers a concrete efficiency objective: retain a pretrained model's behavior within a smaller model storage budget. We study whether coding agents can carry out this step autonomously. We formulate compression as a terminal task in which an agent receives a source checkpoint, storage limit, inference environment, and time budget, then constructs compressed weights and executable inference code. The agent controls its algorithm and experiments; perplexity feedback guides search, and independent verification checks the final model. Across six agents and six model--size settings, 29 of 36 submissions pass verification. Selecting the lowest-perplexity agent design per task yields an average 3.4% perplexity increase over BF16 and 77.33% downstream accuracy, compared with 79.56% for BF16. Recorded experiments reveal how agents achieve these outcomes and where improvement breaks down. Agents lower PPL in 21 of 24 revisions at nearly fixed storage; complete-model feedback reverses 17 of 79 prior local rankings; and generation and inference tests expose failures that likelihood alone misses. The results identify representation design, experimental judgment, and capability validation as central to agent-driven model compression, a measurable component of efficiency-oriented self-improvement.

arXiv:2611.12477v1cs.CV

Perturb, Aggregate then Reconstruct: Consensus Aggregation and Fourier-Attended Multi-Decoders for Reconstruction-based Multi-View Anomaly Detection

Multi-view anomaly detection is crucial for industrial inspection to identify defects hidden from single viewpoints. Yet, despite their strong success in single-view applications, reconstruction-based methods remain largely underexplored in multi-view settings. A key challenge in extending these methods lies in generating view-consistent pseudo-anomalies. To address this, we propose PAR, a reconstruction-based framework for enhanced multi-view anomaly detection. First, we introduce a feature perturbation strategy coupled with a consensus aggregation module that integrates these perturbed features across viewpoints to mitigate view-inconsistent anomalies and construct highly aligned pseudo-anomaly representations. Second, we employ a view-specialized multi-decoder architecture; by assigning a dedicated decoder to each view, we increase the model's representational complexity, enabling the generation of highly feasible, localized anomaly maps rather than less precise, view-agnostic reconstructions. Last, we incorporate Fourier attention into the decoders to emphasize global frequency-domain structures over local token evidence, effectively preventing identity mapping. Extensive experiments on the Real-IAD dataset demonstrate that PAR consistently outperforms state-of-the-art methods in sample-level, image-level, and pixel-level anomaly detection. Our code is available at https://github.com/SpatialAILab/PAR_mvad.

arXiv:2611.11320v1cs.CV

Beyond Similarity Entropy: A Rank-Guided Data-Free Quantization Framework for Vision Transformers

Vision Transformers (ViTs) have demonstrated remarkable performance, yet their computational and memory demands pose significant challenges for deployment on mobile devices. Data-free quantization (DFQ) enables the compression of ViTs without access to their original training data. Existing DFQ methods often synthesize calibration samples by maximizing the entropy of patch similarity. However, similarity entropy describes the dispersion of similarity values without ensuring that the generated samples cover diverse feature directions, potentially limiting their effectiveness for quantization calibration. To address this limitation, we propose RankQ-ViT, a rank-guided DFQ framework for ViTs. We first observe that patch similarity matrices derived from Gaussian noise tend to have lower ranks than those derived from real images. Motivated by this observation, we introduce a block-aware self-contrastive objective that promotes feature diversity during synthetic data generation without explicitly computing matrix rank at each optimization step. We then select calibration samples based on their rank and semantic consistency and use them for block-wise reconstruction of the quantized model. Experiments on ImageNet, COCO, ModelNet40, and ShapeNet demonstrate the effectiveness of RankQ-ViT across multiple architectures and tasks, particularly under low-bit quantization. The code is available at https://github.com/Runqing-forMost/rankq-vit.git

arXiv:2611.16590v1cs.AI

AstroMemBench: A Benchmark for Memory Systems in Embodied Spacecraft Agents

Long-term spacecraft autonomy requires embodied agents to use operational history under communication delays, time-critical operations, environmental uncertainty, complex physical interactions, power and computational constraints, and safety and reliability requirements. We introduce AstroMemBench and a Scenario-Challenge-Capability design method that maps these requirements to executable memory tests. The benchmark covers 16 memory subchallenges through 140 templates and 2240 instances across nine tasks in planetary surface operations, on-orbit servicing, and deep-space exploration. A deterministic environment scores actions for task completion, state changes, safety, and resource use. Ablations under nine conditions show that the benchmark is sensitive to how accurately agents recover and use historical information and to differences in memory-system design. Main experiments compare six configurations. Mem0, Chain-of-Memory, LightMem, and retrieval-augmented generation (RAG) improve same-session mean scores over native Pi Agent, with no clear advantage of the three memory systems over RAG. After a session restart, Mem0 outperforms RAG by 12.25 points. Benefits favor instruction retention and result reuse over ongoing state tracking and energy reservation. AstroMemBench supports the design, development, and selection of memory systems for embodied spacecraft agents. AstroMemBench 1.0 and its evaluation code are publicly available: https://github.com/flyingView/AstroMemBench-v1.

arXiv:2606.17606v1cs.CV

Flux-Guard: 확산 모델을 활용한 얼굴 신원 보호

Flux-Guard: Facial Identity Protection using diffusion models

얼굴 인식(FR) 시스템의 광범위한 배포는 소셜 미디어와 공개 플랫폼에 공유된 개인 이미지를 신원 연결 및 프라이버시 위험에 노출시킵니다. 기존의 적대적 프라이버시 보호 방법은 승인되지 않은 FR 성능을 저하시킬 수 있지만 생성적 얼굴 편집과 호환되지 않습니다. 인공지능 기반 얼굴 편집 도구가 인기를 얻고 있어, 개인화된 초상화 생성 및 소셜 공유에 대한 사용자 수요가 크게 증가했습니다. 그러나 현재 편집 방법은 종종 신원 특징을 보존하여, 편집된 이미지가 악의적인 FR 시스템에 의한 추적에 여전히 취약하게 만듭니다. 따라서 본 논문은 Flux-Guard를 제안합니다. 이는 적대적 공격에 기반한 프라이버시 보호 얼굴 편집 프레임워크로, 통합된 생성 과정 내에서 얼굴 편집과 프라이버시 보호를 결합합니다. 구체적으로, 의미적 조작을 생성 과정과 정렬하기 위한 흐름 궤적 제어 방법을 설계하고, 시각적 품질을 유지하면서 공격 효과를 극대화하기 위해 적대적 강도를 동적으로 조정하는 적응형 지각 손실(perceptual loss) 기반 가중치 전략을 활용한 잠재 공간 적대적 최적화를 도입했습니다. 광범위한 실험을 통해 Flux-Guard가 얼굴 편집을 지원하면서 CelebA-HQ 및 LADN 데이터셋에서 교차 도메인 얼굴 인식 모델에 대한 공격 성공률을 크게 향상시킴을 입증했습니다. 또한, 상용 API에 대한 평가 결과는 실제 애플리케이션에서의 효과를 확인했습니다. 코드는 https://github.com/JLMWang/Flux-Guard에서 공개되었습니다.

arXiv:2606.28367v1cs.IR

Beyond the Reranker: Do RAG Retrieval Enhancements Help Once a Strong Reranker Is Present?

Retrieval-augmented generation (RAG) is routinely extended with methods meant to improve retrieval: query expansion, hierarchical and cross-document summarization, graph-based expansion, per-query routing, rank fusion, and corrective re-retrieval. The benefits reported for these methods come almost exclusively from homogeneous corpora, predominantly Wikipedia prose. Whether they hold on the mixed-format collections common in practice, where code, markdown, tables, scientific PDFs, and prose are interleaved within one corpus, has not been measured. To study this directly, we build \textbf{HetDocQA}, a heterogeneous benchmark with \emph{chunker-agnostic} span-overlap relevance labels and collection-disjoint splits, and pair it with MuSiQue and QASPER as homogeneous controls. We evaluate eight methods on a shared backbone, with bootstrap confidence intervals and multiple-comparison correction. A strong cross-encoder reranker accounts for most of the pipeline's quality; beyond it, only two methods yield reliable gains: query expansion and SSCC. SSCC, a per-source calibrated corrector introduced here, sets a separate acceptance threshold for each score source and helps only on heterogeneous data. The remaining reranking and pool-expansion methods in common use, among them hierarchical summarization, graph expansion, routing, and rank fusion, give no reliable gain once that reranker is present.

arXiv:2606.31029v1cs.CV

TerraDiT-$Ω$: Unified Spatial Control for Satellite Image Synthesis with Any Geospatial Primitive

Generative models have achieved remarkable progress, yet applying them to satellite imagery remains challenging. Unlike natural imagery, satellite scenes are structured by spatially complex and semantically distinct geometries. Prior work addresses this complexity by adapting natural image frameworks using dense rasters or sparse prompts, trading off annotation cost and fidelity while breaking compatibility with vector primitives commonly used to represent geographic information. We introduce TerraDiT-$Ω$, a unified spatial control framework that generates satellite imagery directly from any native geospatial primitive. By jointly leveraging precise annotations (polygons, polylines) and coarser ones (bounding boxes, points), the model supports controllable layouts across varying annotation budgets, broadening applicability to design tasks such as urban planning while remaining naturally compatible with end-to-end GeoAI workflows. To effectively leverage these primitives during generation, we propose Geometry-Aware Local Attention, a conditioning mechanism that injects explicit geometric cues into the attention space. Across all conditioning formats, our approach consistently outperforms both dense-control and sparse-control baselines. Furthermore, this flexibility enables controllable synthetic data augmentation using a single generative model, improving downstream performance on land-cover segmentation, object detection, road graph extraction, and scene classification. Code, data, and weights are available at https://github.com/mvrl/TerraDiT.

arXiv:2607.18026v1cs.AI

이질적 LLM 병합 재고: 가중 모델 평균화 관점

Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective

매개변수 공간이 실질적으로 다른 대규모 언어 모델(LLM)을 학습이나 의미적 정렬(semantic alignment) 없이 직접 가중 평균으로 병합할 수 있는가? 기존 이질적 융합 방법은 일반적으로 지식 증류(distillation), 어댑터(adapter), 학습된 잠재 공간, 라우팅(routing), 또는 특징 정렬(feature alignment)을 도입하며, 진정으로 다른 수십억 매개변수 체크포인트에 더 단순한 방법이 통할 수 있는지는 열린 질문으로 남겨 둔다. 우리는 학습 없는 차원 적응(dimensional adaptation)과 비율 제어 보간(ratio-controlled interpolation)을 통해 이 직관에 반하는 질문을 재검토한다. 합집합(union) 스타일 병합에서는 더 작은 모델을 더 큰 매개변수 공간으로 확장하고, 교집합(intersection) 스타일 병합에서는 더 큰 모델을 더 작은 매개변수 공간으로 잘라낸다. Qwen 계열 모델 쌍과 수학적 추론, 코드 생성, 언어 이해, 상식 추론, 지식, 지시 따르기(instruction following)를 아우르는 벤치마크에 걸쳐, 결정론적 확장은 대체로 원본 모델 기능을 보존하며, 소규모 비율 보간은 상보적 역량을 전이함으로써 강력한 원본 체크포인트보다 나아질 수 있다. 그러나 거의 균형에 가까운 보간은 종종 붕괴하며, 과제 수준 결과는 일부 역량에서의 이득이 다른 역량에서의 퇴보와 공존하는 시소(seesaw) 효과를 드러낸다. 이 결과는 경량 차원 적응과 신중하게 제어된 비율과 결합된 단순 매개변수 평균화가 이질적 LLM 병합을 위한 놀랍도록 강력한 기준선임을 보여주며, 직접 가중 융합의 한계가 더 복잡한 이질적 병합 방법이 대규모로 달성할 수 있는 것의 상한을 제한할 수 있음을 시사한다.

arXiv:2607.19364v1cs.AI

대규모 언어 모델의 활성화 공간 제어를 위한 통계적 근거 기반 희소 특징 개입

Statistically Grounded Sparse-Feature Interventions for Activation-Space Control in Large Language Models

활성화 조향(activation steering)은 대규모 언어 모델(LLM)의 행동 제어를 위한 미세조정의 경량 대안을 제공하지만, SAE 기반 조향 방법은 종종 학습된 조향 목표나 단일 기준 특징 선택에 의존한다. 우리는 먼저 6가지 조건의 신뢰성 필터를 적용한 뒤, $F$-test, KSG 상호정보(mutual information), Cohen's $d$라는 세 가지 상보적 통계량에 대한 비가중 Borda 합의를 통해 희소 특징을 순위화하는 투명한 SAE-특징 조향 파이프라인을 제안한다. 그 결과로 얻어진 조향 방향은 SAE 디코더 행들의 Cohen's $d$ 가중 결합으로 구성되며, 근사적 SAE-특징 비상관(decorrelation) 하에서 Fisher-LDA에 의해 동기화된 최적화 없는(optimization-free) 방향을 제공한다. Gemma 계열 모델 3종, 4개 행동 도메인, 356개 층-강도 구성에 걸쳐, 이 방법은 측정 가능한 도메인별 변화를 만들어내면서 원시 속성(attribute) 이동과 품질을 보존하는 생성 사이에 상당한 격차가 있음을 드러낸다. 가장 강한 구성에서 논리적 정확성(logical-correctness) 조향은 Gemma~2 9B에서 주요 점수(primary-score) 델타 $+1.16$에 도달한다. 그러나 더 넓은 범위의 발견은, 실용 가능한 조향이 모델, 도메인, 층, 강도에 따라 매우 국소화되어 있음을 보여준다. 이 결과는 활성화 조향 평가가 원시 행동 변화와 함께 품질 조건부 성공(quality-conditioned success)을 보고해야 함을 시사한다. 코드와 데이터는 https://github.com/Oshayer-Siddique/LLM-Steering-Using-SAE 에서 이용할 수 있다.

arXiv:2608.01667v1cs.AI

TCPO: 턴 수준 신용 정책 최적화

TCPO: Turn-Level Credit Policy Optimization

검증기(verifier) 기반 강화학습(RL)은 대규모 언어 모델(LLM)의 추론 능력을 향상시키는 강력한 패러다임으로 자리 잡았습니다. 다중 턴(multi-turn) 환경에서 모델은 각 턴 이후 검증기 점수를 수신하고 출력을 반복적으로 개선합니다. 이러한 점수는 조밀한 피드백을 제공하지만, 조밀한 신용(credit)을 직접적으로 제공하지는 않습니다. 점수는 현재 출력의 품질을 측정하는 반면, 신용은 현재 턴이 개선 궤적을 어떻게 변화시키는지를 측정해야 하기 때문입니다. 본 논문에서는 검증기 기반 다중 턴 강화학습을 위한 턴 수준 신용 할당 방법인 TCPO를 제안합니다. TCPO는 신용 할당을 '점수-신용 변환'으로 공식화하고 기준 기반 비교를 통해 턴 수준 어드밴티지(advantage)를 구성합니다. 회고적 신용(retrospective credit)은 최선의 이전 상태 대비 즉각적인 진전 및 후퇴를 포착하고, 사후 지연 신용(hindsight delayed credit)은 추후 보상으로 이어지는 개선되지 않은 턴을 식별하며, 선택적 고정 히스토리 반사실적 추정(selective fixed-history counterfactual estimation)은 동일한 히스토리 하에서 의외성(surprisal)이 높은 턴을 정제합니다. 수학 추론, 코드 생성, AppWorld 에이전트 작업에 대한 실험 결과, TCPO는 모델 규모, 작업 도메인, 검증기 유형 전반에서 가장 강력한 베이스라인들과 동등하거나 개선된 성능을 보였습니다. TCPO는 Qwen3-4B 및 DeepSeek-R1-Distill-Llama-8B에서 최고 또는 공동 최고의 최적 턴 Pass@8을 달성하고, 성공까지의 턴 수를 줄이며, 다중 턴 에이전트 성능을 향상시킵니다. 이러한 결과는 점수-신용 변환이 검증기 기반 다중 턴 정책 최적화의 핵심 요소임을 보여줍니다.

arXiv:2608.14585v1cs.AI

Euclid-Omni: 평면 기하학을 위한 통합 신경-기호 프레임워크

Euclid-Omni : A Unified Neuro-Symbolic Framework for Plane Geometry

유클리드 기하학은 직관적인 도형 이해, 공리적 연역, 대수적 계산의 결합을 요구하므로 AI 추론을 위한 매력적인 시험대이다. 그러나 기존 접근법은 대부분 이러한 능력 중 일부만 다루거나 대회 수준 문제에서 어려움을 겪는다. 우리는 형식 기하 시스템을 대규모 언어 모델(LLM) 및 비전-언어 모델(VLM)과 결합하여, 올림피아드 수준 난이도까지 형식 언어와 자연어 모두에서 계산형 및 증명형 문제를 다루는 통합 신경-기호 프레임워크인 \textit{Euclid-Omni}를 소개한다. 핵심적으로, 우리는 연역적 추론과 대수적 계산을 통해 추론 단계를 자동 생성하는 다재다능한 기호 기하 솔버인 \textit{Euclidea}를 개발한다. 이를 바탕으로 기호 문제와 해를 합성하고, 도형을 렌더링하며, 이를 자연어로 번역하는 데이터 생성 파이프라인을 구축하여, 다양한 추론 설정 전반에 걸친 LLM 및 VLM 학습용 대규모·다양한 데이터셋을 만든다. 실험 결과, 우리의 합성 데이터로 학습한 VLM은 계산 과제에서 우수한 성능을 보였으며, \textit{Euclidea}와 결합한 LLM은 연산량과 학습 데이터를 자릿수 차원에서 훨씬 적게 사용함에도 불구하고 올림피아드 수준 증명 문제에서 최첨단 시스템과 견줄 만한 성능을 달성한다. 코드와 스크립트는 https://github.com/20171130/Euclid-Omni 에서 공개되어 있다.