PyTorchKR Papers

업데이트된 논문

총 15편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2610.00224v2cs.AI

Build2SPARQL: 빌딩 지식 그래프 쿼리를 위한 대규모 Text-to-SPARQL 벤치마크 데이터셋

Build2SPARQL: A Large-Scale Text-to-SPARQL Benchmark Dataset for Building Knowledge Graph Querying

빌딩 자동화 시스템은 Brick 및 ASHRAE 223P와 같은 온톨로지를 사용하는 시맨틱 지식 그래프(KG)로 점점 더 많이 표현되고 있으며, 이는 인공지능 응용을 위한 기계 판독 가능한 기반을 형성합니다. 유망한 응용 분야 중 하나는 자연어 질문을 SPARQL로 변환하는 Text-to-SPARQL로, 빌딩 관리자가 언어 에이전트를 통해 이러한 그래프를 쿼리할 수 있게 해주지만 대규모 자연어/SPARQL 벤치마크의 부족으로 발전이 제한되고 있습니다. 본 논문에서는 지식 그래프 기반(KG-grounded) 파이프라인으로 생성된 빌딩 지식 그래프용 대규모 벤치마크인 Build2SPARQL을 제시합니다. 이 파이프라인은 SPARQL 쿼리를 전적으로 그래프 순회 코드로 생성 및 검증하는 반면, 대규모 언어 모델(LLM)은 자연어 질문만을 생성하도록 하여 쿼리의 정확성을 모델의 동작과 독립적으로 유지합니다. 해당 파이프라인은 선형 체인, 분기, UNION, 집계, OPTIONAL, 속성 필터링의 6가지 쿼리 패턴 패밀리를 추출하며, 각 쿼리를 5가지 어휘 레지스터에 걸쳐 표현합니다. 201개의 빌딩 지식 그래프(Brick 180개, ASHRAE 223P 21개)에 적용하여 6,136개의 실행 가능한 SPARQL 쿼리와 30,680개의 질문을 도출했습니다. 300개 질문에 대한 2인 평가자 인적 검증 결과 98.8%의 시맨틱 충실도, 98.8%의 자연스러움, 84.0%의 운영 타당성을 확인했습니다. 3개의 공개 가중치(open-weight) 언어 모델에 걸친 검색-증강 평가에서는 완전 일치(exact-match) 정확도가 0.2~20%(제로샷)에서 56~65%(3샷 검색)로 향상되었습니다.

arXiv:2601.00821v5cs.AI

구조보다 충실도: 장기 대화 LLM 메모리에서 원문 청크가 손실성 아티팩트 추출을 능가한다

Fidelity Before Structure: Verbatim Chunks Beat Lossy Artifact Extraction in Long-Conversation LLM Memory

A growing class of conversational-memory systems compresses dialogue history into structured artifacts (extracted facts, decisions, or events). The premise is that distilled structure retrieves better than raw text. We test this premise with a controlled ablation. Within one fixed retrieval--rerank--reasoning pipeline, we swap only the stored representation (LLM-extracted typed artifacts versus verbatim conversation chunks) and hold the model, retriever, reranker, and judge constant. Verbatim chunks win by 15.9 points on LoCoMo (43.9% vs. 28.0%) and by 22.0 points on LongMemEval-S (67.4% vs. 45.4%). A semantic graph does not recover the gap, and six confound controls reproduce the effect. The mechanism is lossy distillation rather than structure per se. Accuracy tracks how much source text survives in the store, and the extracted-artifact pipeline does not beat naive RAG in overall accuracy (chunks do abstain worse; see Limitations). For the extraction designs we test, structured memory should augment verbatim text rather than replace it. Adding artifacts alongside chunks costs no detected accuracy, while substituting them forfeits the gap. Code and data are available at https://github.com/tao-hpu/cog-canvas.

arXiv:2607.03836v2cs.CV

단순할수록 좋을 때: 중세 라틴어 필사본을 위한 번역 파이프라인 평가

When Simpler Is Better: Evaluating Translation Pipelines for Medieval Latin Manuscripts

Despite remarkable progress in machine translation, Vision Language Models (VLMs) struggle on historical manuscripts, a domain that stresses core Natural Language Processing (NLP) capabilities: low-resource transliteration, archaic vocabulary, and noisy input signals. We present a systematic framework for evaluating the full image-to-translation pipeline on medieval Latin manuscripts, a setting in which scribal shorthand, ligatures, and parchment degradation expose failure modes that are invisible in clean-text benchmarks. Benchmarking on the CATMuS Latin dataset reveals a specialization gap: domain-specific Optical Character Recognition (OCR) models reduce character error rate by up to 4.3$\times$ compared to general-purpose VLMs, despite operating at orders of magnitude fewer parameters. We introduce the Interpres-Parallel-Corpus (IPC), a novel dataset comprising 1,383 aligned manuscript image lines, transcriptions, and expert translations, to our knowledge the first of its kind for medieval Latin. Our experiments uncover a complexity paradox: the simplest pipeline, a specialized OCR model feeding directly into a VLM, outperforms all multi-component variants. Adding retrieval-augmented generation (RAG) or post-OCR correction introduces prompt saturation and error propagation that degrade aggregate translation quality. These findings offer both a new benchmark and practical guidance for deploying translation systems in low-resource historical settings.

arXiv:2607.05479v2cs.CR

생성형 AI 워크플로우에서의 특권과 기밀성

Privilege and confidentiality in generative AI workflows

생성형 AI(GenAI) 시스템은 클라이언트 데이터를 세 가지 방식으로 저장하고 처리한다: 학습과 기억화를 통해 모델의 파라미터에, 실시간 세션 중 컨텍스트 윈도우에, 그리고 검색-증강 생성(RAG)을 위한 지식 데이터베이스에. 각 방식은 기밀성과 법적 직업상 특권에 대해 서로 다르고 종종 반직관적인 위험을 초래하며, 각각 특정한 거버넌스 대응을 요구한다. 특권과 생성형 AI를 다룬 최초의 영국 및 미국 판결인 UK and Munir v Secretary of State for the Home Department와 United States v Heppner, 이 판결들이 해석되어야 할 기존의 특권 판례들, 그리고 최근의 컴퓨터 과학 연구를 바탕으로, 우리는 실무자들이 이해할 수 있는 방식으로 데이터 저장 및 처리의 세 가지 방식을 설명하고 각각의 법적 결과를 분석한다. 이어서 우리는 이 분석을 잉글랜드 및 웨일스의 사무변호사를 규율하는 규제 체계와 통상적인 직업상 과실의 일반 원칙 내에 위치시키며, 효과적인 정보 거버넌스의 기준(그리고 그에 따라 과실 및 위법 행위를 판단하는 벤치마크)이 변화하고 있다고 주장한다. 우리는 주로 SRA 규제를 받는 실무자들을 대상으로 서술하지만, 데이터 거버넌스 분석은 특권 또는 직업상 비밀의 보호가 증명 가능한 기밀성에 달려 있는 모든 법적 관할권으로 확장될 수 있도록 구성하였다. 이 논문의 궁극적인 목적은 법률 서비스 전문가들이 GenAI 시스템에서의 주요 데이터 유출 위험을 이해하도록 돕고, 이를 통해 클라이언트 데이터 및 기타 민감한 자료에 대한 GenAI의 보다 책임 있는 배포를 촉진하는 것이다.

arXiv:2604.22294v3cs.CL

SLIDERS: Evidence Synthesis with Autonomous Database Reconciliation for Systematic Reviews

Systematic reviews requires comprehensive evidence synthesis from large document corpora to answer targeted research questions and are foundational in finance, social sciences, and other technical fields. Manual construction of evidence tables is labor-intensive, and recent LLM-based assistants relying on embedding or keyword search often fail to meet the coverage standards of systematic reviews. We introduce SLIDERS, a novel LLM-based methodology for automatically assembling evidence tables tailored to research questions for systematic reviews. SLIDERS extracts full-text excerpts that serve as direct evidence or as provenance for structured data. Core to SLIDERS is an autonomous evidence reconciliation coding agent that iteratively writes SQL to analyze and reconcile extracted evidence, bringing together information fragmented across documents, resolving inconsistencies and removing duplicates across records. We evaluate SLIDERS on BrowseComp+, achieving 90% accuracy, and show that it outperforms all baselines on additional benchmarks within the LLM's context window. To assess scalability, we introduce two new evidence-synthesis benchmarks, WikiCeleb100 (3.9M tokens) and FinQ100 (36M tokens), which include follow-up questions beyond the original research query. Although SLIDERS synthesizes evidence knowing only the original question, it answers 78.9% and 55.2% of all questions correctly on WikiCeleb100 and FinQ100, respectively

arXiv:2505.13339v2cs.RO

OPA-Pack: Object-Property-Aware Robotic Bin Packing

Robotic bin packing is a fundamental manipulation problem with broad applications in e-commerce and warehouse automation. However, existing methods primarily focus on object geometry to optimize packing compactness without explicitly modeling object properties such as fragility, edibility, and chemical compatibility that humans typically consider during packing. This paper presents OPA-Pack (Object-Property-Aware Packing), a framework that incorporates object-property information into robotic bin-packing planning. The framework includes an object property recognition scheme based on retrieval-augmented generation and chain-of-thought reasoning to establish a dataset with object property annotations for 1,032 everyday objects. The framework further incorporates OPA-Net, which encodes candidate-object properties via a property embedding layer and represents the container state using fragility and avoidance heightmaps. We design a reward function and adopt a deep Q-learning scheme to train OPA-Net. Experimental results show that OPA-Pack improves avoidance accuracy from 52.3% to 95.0%, reduces load on fragile objects by 29.4%, and achieves a packing compactness of 0.413, compared with 0.420 for the baseline. In addition, we evaluate OPA-Pack on a physical platform using 70 packing sequences sampled from 100 scanned real-world objects. Results show that OPA-Pack outperforms the baseline and other packing heuristics in reducing the number of closely packed incompatible object pairs and the number of compressed fragile objects while preserving packing compactness.

arXiv:2606.11337v2cs.AI

Can AI Agents Synthesize Scientific Conclusions in Health?

Scientific AI agents increasingly retrieve evidence, reason across sources, and synthesize conclusions used in consequential decisions. Yet, their ability to do so in high-stakes domains such as health remains unclear. We introduce SciConBench, a large-scale live benchmark of 9.21K questions and expert-written conclusions from systematic reviews to evaluate open-domain scientific conclusion synthesis in health and medical domains. The benchmark draws on an expert-validated automated evaluation pipeline that decomposes conclusions into atomic facts and measures correctness and comprehensiveness via factual precision and recall. To mitigate data leakage, we further introduce SciConHarness, a clean-room evaluation harness that equips agents with controlled web interaction to ensure valid measurement. Evaluating 8 frontier models and deep research agents, we find that factual quality remains low: under clean-room settings, the best agent achieves only a factual F1 of 0.337. Our clean-room setting consistently reduces performance relative to unconstrained evaluation, suggesting that leakage inflates estimates of models' true synthesis capabilities. Finally, we audit consumer-facing agents (e.g., Google AI Overview, OpenEvidence) and find they frequently generate incomplete and sometimes contradictory conclusions, even when the ground-truth answer is available. Overall, our results show that reliable synthesis of scientific conclusions in health remains an open challenge, and that clean-room evaluation is essential for assessing open-domain AI agents.

arXiv:2607.24743v4cs.CV

ClinFusion: 포괄적 의료 이해를 위한 비전 중심 다중모달 LLM 시스템

ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding

다중모달 대규모 언어 모델(MLLM)은 임상 실무를 혁신할 막대한 잠재력을 지니고 있으나, 의료 분야에 이를 배포하는 것은 근본적으로 비전 중심의 과제이다. 모델은 이질적인 2D 및 3D 의료 영상으로부터 지식을 흡수해야 하며, 평가 프로토콜은 방사선 전문의의 임상 실무와 부합해야 하고 정확하고 세밀하며 사실성 중심의 평가를 제공해야 한다. 본 논문에서는 이러한 한계를 체계적으로 해결하기 위해 포괄적 의료 이해를 위해 설계된 비전 중심 MLLM인 ClinFusion을 소개한다. 우리는 Cascade Spatial-Aware Locality Fusion 연산자를 특징으로 하는 구성적·계단식 비전 인코더 아키텍처를 제안하며, 이는 융합 인코더 내에서 다양한 2D 및 네이티브 3D 의료 영상 이해를 통합한다. 또한 지시 따르기(instruction following) 평가를 위한 MedIF-Bench와, 임상적으로 정렬되고 사실성 중심의 보고서 생성 평가를 위한 관심 영역(RoI) 기반 방법을 포함하는 비전 그라운딩(vision-grounded) 평가 프레임워크를 추가로 도입한다. ClinFusion이 시각적 질의응답, 보고서 생성, 지시 따르기에 이르는 2D 및 3D 다중모달 의료 벤치마크와 텍스트 의료 과제를 아우르는 포괄적 벤치마크 모음 전반에서 새로운 최첨단(state-of-the-art) 성능을 달성함을 보인다. 24개 벤치마크 중 20개에서 Hulu-Med, Lingshu 등 선도적인 오픈소스 의료 MLLM을 능가했으며, 16개 벤치마크 중 13개에서 GPT-5.2, Gemini-3-Flash와 같은 강력한 독점 모델보다 우수한 다중모달 능력을 보였고, 검색-증강 생성(RAG) 및 도구 지원 임상 워크플로를 위한 에이전트(agent) 도구 사용으로 더욱 확장할 수 있다. 방사선 전문의 자격을 갖춘 전문의들의 단맹 평가는 ClinFusion이 최상위 순위의 보고서를 생성함을 확인했으며, 우리의 RoI 기반 지표가 검토된 모든 자동 평가 지표 중 전문가 판단과 가장 강한 상관관계를 달성함을 검증했다.

arXiv:2607.09489v3cs.AI

Ceci n'est pas une pipe: A semantics of justified reliance for AI systems

An AI agent may keep a successful check in context after an edit has invalidated its evidence. We give a semantics for specifying when a system may rely on a message and act on it. Reliability checks an asserted claim against accepted domain knowledge, sources, and messages in the selected context. Justified reliance additionally checks current evidence, authority, and the proposed world change. Relational component semantics and event traces represent nondeterministic outputs, iteration, and external changes. We characterise when merging claims preserves derivable conclusion classes, then construct the coarsest permitted quotient whose rules preserve the exact statuses of designated concrete claims. Construction is polynomial for unary rules; deciding even one permitted merge is coNP-complete for acyclic binary rules. This applies completeness and strong-preservation methods to the execution of quotiented rules. We also relate transition safeguards to runtime enforcement. Relative to a supplied set of possible worlds, a sound guard accepts an observation only when every matching world passes the checks. Exact acceptance requires observations that distinguish opposite-decision worlds; choosing sufficient services is NP-complete on explicit finite tables. A complete finite passport-renewal instance, Lean theorem encodings, Z3 instance checks, and an OCaml harness exercise the semantics. Domain knowledge, interpretation, source licenses, and policy remain application inputs.

arXiv:2606.12616v2cs.AI

PersonaDrive: 폐루프 주행 시뮬레이션을 위한 인간 스타일 검색-증강 VLA 에이전트

PersonaDrive: Human-Style Retrieval-Augmented VLA Agents for Closed-Loop Driving Simulation

Closed-loop driving simulators mostly fill their environments with traffic agents that behave alike, whether they are rule-based or learned. Recent methods that control driving style learn it from labels inferred from existing driving recordings, from synthetic data generated to reflect each style, or from reward weights estimated by a large language model (LLM). All of these are indirect estimates of a style, not recordings of people who were instructed to drive in it. We introduce PersonaDrive, a vision-language-action (VLA) driving agent that takes its style from retrieved human demonstrations. The demonstrations come from a new dataset in which 15 participants drove CARLA routes on a driving simulator under conservative, neutral, and aggressive instructions. PersonaDrive trains a lightweight retrieval head on features from a frozen image encoder and on control signals over per-style databases, and fine-tunes one VLA backbone over the retrieved demonstrations. Switching style only means searching a different FAISS index, with no retraining. On the Bench2Drive benchmark without style conditioning, PersonaDrive improves the Driving Score by 4.6% over SimLingo and by 2.5% over HiP-AD, a gain that fine-tuning without retrieval does not reproduce. With style conditioning, PersonaDrive outperforms the strongest baseline, Drive My Way, in every style, with a Driving Score 6.7% to 11.6% higher. Its driving also follows the instruction, since average speed rises by 19.8% from the conservative to the aggressive style. Each traffic agent is evaluated individually as a driving policy. The dataset and model weights are available on the project page: https://pervasiveautonomylab.github.io/personadrive-landpage/.

arXiv:2505.23277v4cs.CL공식 코드 · Papers with Code

Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compression

Retrieved contexts are often long, noisy, and redundant, motivating query-aware compression before answer generation. Native attention provides signals for context selection, but heterogeneous head behaviors complicate their direct aggregation into sentence-utility scores. We introduce Sentinel, a sentence-level compressor that learns to combine these signals through a signed linear readout across layers and heads. Only the readout is trained using existing QA evidence annotations, while the backbone remains frozen and sentence scoring requires no autoregressive generation. Across twelve LongBench tasks, Sentinel with a 0.5B backbone outperforms Full Context on multiple tasks under a 2,000-token budget with each of three readers: Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, and DeepSeek V4.1 Flash. Despite English-only QA supervision, the same readout transfers to Chinese tasks without task-specific retraining. Length-stratified profiling further shows lower compression latency and GPU memory usage than LLMLingua-2 and CPC.

arXiv:2509.00761v5cs.AI

L-MARS: 에이전틱 검색 및 인용 충실성 감사를 갖춘 법률 다중 에이전트 시스템

L-MARS: Legal Multi-Agent System with Agentic Search and Citation-Faithfulness Audit

대규모 언어 모델(LLM)은 법률 질의응답에 점점 더 많이 배포되고 있으며, 평가는 일반적으로 객관식 정확도에 초점을 맞춘다. 이 지표는 흔한 실패를 간과한다. 즉, 답변에 붙은 인용 출처가 실제로 존재하는지, 그리고 시스템이 그 출처에 귀속시킨 규칙을 실제로 뒷받침하는지 여부이다. 우리는 에이전틱 검색과 심사(judge) 기반 증거 검증을 갖춘 공개 다중 에이전트 법률 QA 시스템인 L-MARS를 제안하고, 인용된 출처에 대해 주장(claim)별로 감사한다. 각 원자적 주장은 6분류 분류체계로 레이블링되며, 답변 생성기와 검증기가 서로 다른 모델 계열에서 나온 교차 제공자(cross-provider) 심사 하에서 strict-ALCE로 채점된다. 층화된 100문항 Bar Exam 감사에서 검색(retrieval)은 정확도를 거의 올리지 못하지만, 다중 턴 심사 루프는 strict citation F1을 0.13(단순 RAG)에서 0.25로 끌어올리고 무인용률을 34%에서 13%로 줄인다. 또한 Faith-Search를 도입한다. 이는 접근 불가능한 인용을 재검증하고 수정하는 초안 후(post-draft) 단계이며, 접근 불가능 비율을 1% 미만으로 낮추지만 다중 턴 루프 대비 F1을 개선하지는 않으므로, 충실성 돌파가 아니라 목표 지향적 접근 가능성(reachability) 개입으로 보고한다. 50문항 LegalSearchQA 사례 연구가 이 결과를 확인한다. 검색-후-초안(retrieve-then-draft) 파이프라인은 인용 F1이 약 0.75 근처에서 포화되지만, 단일 에이전트 웹 검색 기준선은 외부 감사 하에서 0.22로 급락한다.

arXiv:2501.08686v2cs.DB공식 코드 · Papers with Code

Knowledge Graph-based Retrieval-Augmented Generation for Schema Matching

Traditional similarity-based schema matching methods fail to resolve semantic ambiguities and conflicts in complex, domain-specific mappings due to a lack of commonsense and domain-specific knowledge. In parallel, large language models (LLMs) without sufficient background knowledge tend to hallucinate, further complicating schema matching. To overcome these limitations, we introduce KG-RAG4SM, a Knowledge Graph-based Retrieval-Augmented Generation for Schema Matching. This method integrates novel retrieval mechanisms, including LLM-based, vector-based, graph traversal-based, and hybrid techniques with advanced reranking schemes to extract the most relevant subgraphs from large external KGs. Our approach achieves superior results without requiring re-training, as evidenced by improvements of 39.79% in precision and 30.56% in F1 score over LLM-based state-of-the-art (SOTA) methods (e.g., Jellyfish-8B) on the MIMIC dataset, and enhancements of 69.20% in precision and 21.99% in F1 score over pre-trained language model (PLM)-based SOTA methods (e.g., SMAT) on the Synthea dataset. Additionally, KG-RAG4SM scales effectively across KGs from commonsense KGs to domain-specific ones, while its instruction-tuning knowledge exclusion component ensures truthful performance in end-to-end schema matching tasks despite the suboptimal relevance of KG context. Our empirical analysis and case studies from real-world scenarios show that the hallucination problem in LLMs for schema matching is well mitigated by our solution.

arXiv:2606.28447v2cs.IR

SemFlowRAG: Directed Semantic Flow from Abstraction to Evidence for Complex Reasoning

Retrieval-Augmented Generation (RAG) enhanced by Knowledge Graphs has shown promise in complex multi-hop reasoning tasks. However, existing graph-based retrieval methods typically rely on flat, undirected topologies. During the retrieval process, the probability flow often gets trapped in high-degree abstract concept nodes which we define as ``probability black holes'', leading to semantic drift and noise accumulation. To address this, we propose SemFlowRAG, a framework that reconstructs the flat retrieval space into a corpus-adaptive semantic gradient graph. This data-driven self-organization enables a hierarchical structure to emerge naturally from the data distribution, capturing the intrinsic semantic granularity of the corpus to suppress structural noise. By quantifying the semantic abstractness of entities through the embedding variance of their associated passages, we transform static undirected edges into directed semantic constraints. Furthermore, we design an abstractness-guided directed PageRank algorithm that forces the retrieval trajectory to follow a ``high-to-low semantic abstractness'' gradient. This mechanism ensures layer-by-layer evidence convergence, smoothly guiding the retrieval process from abstract concepts to specific document evidence. Extensive experiments on complex QA datasets demonstrate that SemFlowRAG effectively mitigates the ``probability black holes'' issue, outperforming existing baselines in both retrieval and downstream reasoning performance.

arXiv:2606.15735v4cs.CL

EHRNote-ChatQA: 종방향 퇴원 요약을 위한 증거 기반 멀티턴 임상 질의응답 벤치마크

EHRNote-ChatQA: A Benchmark for Evidence-Grounded Multi-Turn Clinical Question Answering over Longitudinal Discharge Summaries

Discharge summaries are crucial clinical documents containing the context of a patient's overall hospital stay, and are routinely reviewed by medical experts for patient readmission, ongoing care, and diagnostic decision-making. When reviewing them, medical experts often must iteratively synthesize information across multiple discharge summaries while verifying the evidence supporting each answer. Although large language models (LLMs) are increasingly explored for clinical question answering, existing benchmarks do not sufficiently reflect this setting: they often evaluate exam-style medical knowledge or focus on single-turn question answering with limited evidence-grounding evaluation. We introduce EHRNote-ChatQA, the first benchmark for evidence-grounded multi-turn clinical question answering over patients' multiple discharge summaries. Built from de-identified MIMIC-IV discharge summaries, EHRNote-ChatQA contains 967 patient-level multi-turn samples spanning one to five notes and 16,072 medical-expert-verified QA pairs (8,036 content questions, each paired with an evidence-grounding question) across eight clinical categories. The benchmark is constructed through an expert-informed pipeline combining discharge-summary structuring schema, expert-curated multi-turn QA templates, and LLM-based generation, followed by review and revision of every single QA sample by 12 medical experts (11 physicians and 1 nurse). Benchmarking 24 proprietary and open-weight LLMs reveals several challenges, including that LLMs struggle more with evidence grounding than content answering, multi-turn errors compound across turns, and single-turn clinical QA performance does not reliably transfer to this setting. These findings establish EHRNote-ChatQA as a rigorous and practical benchmark for evaluating clinical QA systems.