대규모 언어 모델 기반 에이전트는 다중 라운드 반성(multi-round reflection)을 통해 단일 태스크 내 실패를 복구할 수 있지만, 각 태스크는 독립적인 에피소드로 남아 있다: 한 태스크에서 여러 반성 라운드를 통해 얻은 교훈은 다음 태스크가 시작되기 전에 사라진다. 우리는 코드 생성 태스크에서 이 격차를 연구한다. 에이전트는 과학 논문의 한 섹션으로부터 오픈소스 Manim 라이브러리를 이용하여 Python 코드를 작성하고 수학적 애니메이션을 렌더링한다. 우리는 ManimAgent를 제안한다. 이는 자기 진화형 멀티모달 에이전트로, 가중치 업데이트나 인간 시드 없이 자체 태스크 스트림으로부터 완전히 성장하는 이중 채널 에피소딕 메모리 뱅크(Episodic Memory Bank)를 통해 태스크 간 반성 경험을 이어간다. 각 애니메이션이 수렴한 후, 비전-언어 모델이 렌더링된 키프레임을 채점하며, 결과 신호는 성공 근거를 소프트 참조 예시(Reference Examples)로 저장하는 긍정 채널 M+와, 검증된 실패 패턴을 하드 기지 함정(Known Pitfalls)으로 저장하는 부정 채널 M-를 채운다. 메모리 없음, 동일 예산의 검색-증강 생성(RAG), 셔플된 메모리 기준선에 대한 고정 프로브 평가에서, 블라인드 인간 평가자의 Pass@1은 상승하고 반성 라운드 수는 메모리 크기가 증가함에 따라 감소한다. 코드, 고정된 메모리 스냅샷, 태스크 스트림을 공개할 예정이다.
비전-언어 모델(VLM)의 시각적 추론 성능을 향상시키려면 일반적으로 재학습이나 수작업으로 설계된 프롬프트 및 도구가 필요합니다. 본 논문에서는 가중치 업데이트 없이 고정된 VLM을 적응시키는 학습-불필요(training-free) 프레임워크인 Dynamo를 제안합니다. 소규모 레이블링된 학습 서브셋에서, 에이전트는 자신의 정답 및 오답 시도를 스스로 검토하고 두 가지 상호보완적 능력을 진화시킵니다: 인지적 병목에 대응하는 재사용 가능한 추론 스킬과, 지각적 병목에 대응하는 실행 가능한 시각 도구입니다. 생성된 각 도구에는 호출 시점을 명시하는 스킬이 짝지어지며, 두 능력 유형 모두 영속적인 라이브러리에 축적됩니다. 네 가지 시각적 추론 벤치마크와 다섯 가지 VLM 백본에 걸쳐, Dynamo는 20개의 모델-벤치마크 설정 전체에서 직접 추론 성능을 향상시켰습니다(평균 +5.6 정확도). 도구 집합이 사전에 주어지는 경우, 본 프레임워크는 각 도구의 호출 시점을 학습하며, 단계별 도구 선택이 테스트된 모든 백본에서 개선되었습니다. 태스크별 강화학습(RL) 방법(VTool-R1, DeepEyes)과 비교할 때, Dynamo는 훨씬 적은 연산량으로 RL 성능 격차의 65~99%를 줄이며, RL과 결합 시 추가적인 성능 향상을 달성합니다.
Chain-of-thought (CoT) prompting improves LLM reasoning, but the source is contested: do the intermediate steps help because they carry useful semantic content, or because conditioning on more tokens buys extra computation before the model commits to an answer? We bring two lines of evidence to bear. First, in distribution: we repeatedly sample each model on the same question and pair a shorter with a longer of its own natural generations that follow the same reasoning plan, so nothing is rewritten and both traces are genuinely in-distribution. Across 25 models the extra tokens leave accuracy essentially unchanged for every independently-trained reasoner, and a blind analysis of the surplus tokens shows that what gain exists elsewhere tracks validation- and checking-content, not verbosity per se. Second, as a controlled intervention, we ask whether two traces expressing the same semantic content (the same facts, operations, and intermediate values, verified through directed acyclic graph equivalence) produce different outcomes when one is more verbose, using a dual-validator design across four targets and eight benchmarks with number-redacted completion and stratified bootstrap confidence intervals. Verbose traces do improve accuracy (25 of 32 benchmark-target cells are positive under at least one validator), but the effects are modest (typically 1-4 points) and depend on the quality of the verbose prose, not merely its length. Under maximum numerical redaction the effect is amplified (median 3.24x across four arithmetic benchmarks), and length-matched non-reasoning filler recovers none of it. Both lines converge: what matters is what the extra tokens do (the reasoning and validation content they carry), not how many there are, a picture neither a pure forward-pass-compute nor a pure semantic-content account fully explains.
Long-horizon strategic planning in complex strategy games demands concurrent reasoning across multiple decision domains under imperfect information and sparse reward. Existing LLM-based agents suffer from three systematic failures: scene blindness from raw tile coordinates, context overflow and domain coupling from monolithic state dumps, and shallow cross-game learning that treats each episode in isolation. We present SAGA, an LLM multi-agent framework with three mechanisms each directly targeting one class of failure: (i) a Map-Semantic Scene Graph that encodes typed spatial relations among game entities into per-unit natural-language context, resolving spatial blindness without global token inflation; (ii) a Tool-Augmented Planner that pulls fine-grained domain state on demand and dispatches per-domain directives to dedicated specialist controllers, eliminating context overflow, domain coupling, and mechanical constraint violations; and (iii) a Dual-Horizon Feedback Loop that combines periodic within-game goal generation with structured cross-game causal post-mortem, enabling principled strategic evolution without manual reward engineering. Evaluated on FreeCiv, SAGA attains the highest mean civilization score -- the environment's sole sparse objective reward -- with lower variance than the two strongest baselines, and is the only method that significantly surpasses every baseline on infrastructure construction, the resource axis most readily sacrificed under multi-objective conflict. It outscores the two strongest baselines in most head-to-head games while cutting output tokens (the dominant decoding cost) by 27%. Equipped with the cross-game evolution module, SAGA reaches the highest end-of-chain score across five successive episodes. Ablation studies confirm that each architectural component contributes independently to this advantage.
Distilling historical trajectories into reusable experience to enhance future problem-solving has become a focal point of recent LLM research. However, existing methods predominantly operate at the task level, leveraging general summaries or rules under the assumption that analogous tasks share universal solution patterns. This approach often fails in complex reasoning, which typically falters at local bottlenecks that require precise, state-specific guidance rather than broad heuristics. We introduce HippoSpark, a state-level experience system that performs on-demand retrieval tailored to the immediate needs of the current reasoning state. Across mathematical, scientific, and programming benchmarks, HippoSpark consistently outperforms both standard prompting and task-level experience baselines. Our findings reveal that the most effective experience systems are those that provide actionable guidance at critical bottlenecks rather than serving as generic task-level context. Our code is available at https://github.com/DanlingMeng/HippoSpark.
In real-world applications, guardrails are often expected to identify unsafe user-model interactions according to application-specific safety policies, rather than relying on predefined risk taxonomies. In this work, we study this setting under the paradigm of in-context policy guardrailing, where guardrails predict safety violations based on policy specifications provided in context. To systematically evaluate this capability, we introduce SafePyramid, a safety benchmark comprising 1,000 multi-turn conversations across 10 domains and 3,000 corresponding application-specific policies, which together contain 61,699 distinct natural-language rules. SafePyramid organizes the evaluation into three difficulty levels: L0 evaluates individual-rule understanding, L1 evaluates reasoning over rule dependencies, and L2 evaluates adaptation of full novel policy frameworks defined in context. To ensure benchmark quality, we employ a rigorous multi-stage pipeline to construct and validate the benchmark. Using SafePyramid, we evaluate 10 frontier LLMs and 5 policy-configurable guardrails and find that in-context policy guardrailing remains highly challenging: even the best-performing model, GPT-5.5, exactly identifies the full set of violated rules in only 54.0%, 35.3%, and 12.9% cases on L0, L1, and L2, respectively. These results highlight the limitations of current guardrails and call for stronger in-context policy guardrails that can reliably execute policies, resolve rule dependencies, and adapt to novel policy frameworks.
SWE-agent established the action interface as an underexplored design axis for software-engineering agents; we make the analogous case for the observation interface in computer-use (CU) agents. Current CU agents, closed and open-source alike, tie observation to action--one screenshot every 3-5 s, no audio--leaving them blind and deaf between screenshots to video, animations, transient UI events, meetings, and spoken instructions. We introduce the Agent-Computer Observation Interface (AOI), a model-agnostic perception layer that decouples continuous, adaptive observation from discrete actions through three gated components: inter-step keyframe capture, volume-gated audio transcription, and CU-model-generated visual narration that persists as text. Each produces almost nothing on static, silent content, reducing to the standard loop without degrading it. On DynaCU-Bench (100 dynamic browser tasks plus a 50-task static control), CU models from 7B to frontier scale gain +17 to +48 pp over their screenshot baselines with zero retraining, turning tasks that are near-impossible from periodic screenshots into largely solved ones. The gap is starkest on audio: on a spoken-content subset AOI agents solve every task, whereas streaming voice models hear accurately but cannot act on what they hear without the scaffold. The decomposition is as informative as the headline gain: keyframe selection turns out not to matter--the value comes from narrating captured frames into persistent text--and the interface is not a fixed bundle, since on a newer model (Gemini 3 Flash) the keyframe stream actively regresses through image-token dilution, so its components must be selected per model rather than shipped as one configuration.
Chain-of-Thought (CoT) improves large language models (LLMs) on difficult reasoning tasks, but it often incurs long natural-language rationales that are poorly aligned with efficient machine reasoning. We propose Communicative Language Symbolism Routing (CLSR), a test-time framework in which multiple LLM agents autonomously invent, evolve, and share compact Language Symbolism Frameworks (LSFs), while a latent-free router adaptively selects and composes these languages per query to optimize the accuracy-token trade-off. Unlike prompt optimization that refines surface instructions, CLSR treats each LSF as a reusable symbolic protocol with compact symbols, usage rules, and a message-passing contract, and improves it through an evolutionary loop driven by correctness and token cost. At inference time, the router may invoke a single low-cost LSF call, ensemble multiple LSFs, or execute a multi-round LSF composition protocol on harder queries. Across challenging benchmarks, CLSR reduces latency-oriented generated token completion by $3\sim 6\times$ compared to standard CoT while maintaining accuracy. We further derive an information-theoretic lower bound on token cost under arbitrary symbolism and show that, under an interpreter-realizability premise, multi-round LSF protocols conditionally subsume program-execution pipelines. Code is publicly available (https://github.com/pzqpzq/LSF_MDia).
Large language models (LLMs) are increasingly used to take actions in the real world and support human decision-making, yet most agents rely on parametric knowledge, fixed post-training data, retrieval, or search. This paradigm breaks down in novel domains and for sophisticated queries that cannot be answered from prior knowledge alone. Knowing the laws of physics, for instance, does not by itself enable LLMs to answer queries or complete long-horizon tasks in a complex physical system. To address this, we introduce Hierarchical Experimentalist Agents (HExA), an in-context self-improvement framework to learn from active experimentation. HExA iteratively designs and refines query-relevant experiments, learns a reusable library of composable skills from experience, and integrates experimental evidence to answer queries or take actions. HExA is training-free, compatible with any black-box model, and does not require external supervision, oracles, or offline data. To evaluate active experimentation, we introduce Interphyre, a tool-calling benchmark built on the PHYRE 2D procedural physics environment, where agents propose interventions and test hypotheses through simulation APIs. Experiments show that current LLM agents struggle in these settings, especially on the hardest levels of Interphyre. Claude Sonnet 4.6 achieves only 2% success, while HExA improves the same model to up to 77% success. HExA also improves open-weight models and outperforms agentic baselines such as ReAct and Reflexion. Moreover, using only skills learned from easier levels and transferred without active experimentation, HExA achieves 44% success, demonstrating the reusability and generalization of its learned skills. Overall, HExA shows that learning through active experimentation can help agents discover useful knowledge, acquire reusable skills, and make efficient progress on novel long-horizon tasks.
We introduce the Complexity Ceiling Benchmark (CCB), a controlled evaluation of how language-model reasoning decays as the number of required sequential steps grows. CCB fixes the semantic content of a task and varies only its depth N in {5,...,50} across three structurally distinct regimes: grounded spatial state-tracking, abstract symbolic pointer manipulation, and transitive relational inference. Across 6,000 trials over five frontier and open-weight LLMs we find a consistent pattern of geometric per-step decay with widely separated domain ceilings: on the first two regimes the strongest models retain pd>0.92 across N=50; on the third every model collapses by N=5, with the best model's 50%-success horizon at H0.5~4.7 steps despite pd=0.863. A trace-level metric (TFBC) shows that 14.5% of correct answers across the benchmark are reached via incorrect intermediate reasoning. Forced verbose state-tracking does not move the ceiling (McNemar p=1.000), and the mean step at which reasoning first diverges, k*, predicts within-domain accuracy better than parameter count. CCB and the geometric decay model together reduce a model's long-horizon reasoning profile to one interpretable number per task family.
Open-ended scientific discovery with large language models (LLMs) increasingly operates as a long-horizon loop of hypothesis search and verification, where a reward signal guides which hypotheses to test next. A notable recent example is AutoDiscovery, which uses "Bayesian surprise" - the belief shift an LLM undergoes after observing evidence for a hypothesis - as both a discovery metric and a reward for search. We first observe that AutoDiscovery treats surprisal as a static quantity, while surprisal in human reasoning is non-stationary - it is defined relative to beliefs that evolve with experience, a prerequisite for continual scientific discovery. We address this mismatch with evidence-informed LLM beliefs: priors updated with evidence from previous hypotheses to compute non-stationary surprisal for new hypotheses. We compare in-context belief-updating mechanisms and find that embedding-based retrieval-augmented generation over prior discoveries best anticipates eventual posteriors, identifying 37.5% of static surprisals as spurious. We then modify search to avoid these spurious rewards and prioritize hypotheses that remain surprising under non-stationary beliefs. Concretely, we introduce two complementary changes to the original search procedure: belief-update filtering and diversity maximization. Across five discovery domains, our method increases accumulated non-stationary surprisal by 30.62% on average compared to the original search procedure, demonstrating that continual scientific discovery with LLMs requires not only better belief measurement but also search procedures that avoid redundancy and encourage diversity.
When does retention matter for memory-augmented LLM agents? We study this with TraceRetain, a lightweight framework for bounded external memory in frozen LLM agents that scores entries by interpretable features (success, age, access frequency, redundancy, specificity, similarity, downstream utility) and evicts the lowest-scoring ones at capacity. On clean ALFWorld with gpt-5-mini, external memory robustly improves over no memory across two seeds, but differences among bounded retention policies fall within Wilson 95% CIs: clean ALFWorld at T=100 to T=200 does not naturally exhibit the memory pollution retention is designed to address. Under a controlled noisy-write stress (75% synthetic distractors), unbounded memory and FIFO-K50 degrade on Precision@5 (20.2% to 12.4% and 15.8% to 3.8%) while TraceRetain-CEM is essentially unchanged (16.9% to 16.6%) and preserves 97/100 task success. The mechanism: unbounded memory has the highest mean similarity (0.87) but lowest precision, indicating failed distractors close to the query in embedding space. Held-out in-distribution evaluation shows memory-augmented policies solving 47 to 49 of 50 tasks vs. 39/50 for no memory. Bounded retention buys memory and step efficiency on saturated clean benchmarks at no task-success cost, and only differentiates from cache heuristics when streams contain noise.
Recent artificial intelligence (AI) systems have shown remarkable progress in mathematical reasoning. Many existing approaches, including large language models (LLMs), draw on human prior knowledge in the form of mathematical text, code, or theorem libraries. Although these approaches are highly effective in practice, it remains an open question whether an agent can autonomously discover useful theorems without such human priors. We study this question in a formal axiomatic system by developing an agent that starts from axioms and inference rules alone and gradually grows a library of useful theorems. Concretely, we propose a self-supervised theorem-discovery algorithm that alternates between proof search and useful-theorem extraction, building a theorem library whose entries are reused as lemmas for subsequent proof search. Experiments show that the agent discovers tens of thousands of theorems and finds proofs for human-written benchmark problems, suggesting that its discoveries include theorems meaningful from a human mathematical perspective. Furthermore, the discovered theorems improve LLM proof performance when provided as prompt lemmas, indicating that they can serve as external knowledge for LLM reasoning. Our results provide evidence that useful theorems can emerge from proof search without relying on human-provided theorem libraries. More broadly, they suggest a path toward self-evolving AI systems for mathematics whose discoveries remain formally verifiable.
LLM agents are increasingly improved without weight updates by evolving a natural-language artifact, such as reflections, workflows, playbooks, cheatsheets, or optimized prompts, that conditions a frozen policy. Such methods are typically reported as wins on the single benchmark where they help. We study them apples-to-apples and surface a sharper picture. We introduce RSEA, a Recursive Self-Evolving Agent that carries a compact three-layer natural-language state: an imperative strategy, reusable skills, and a procedural playbook. Across generations, RSEA rewrites all three layers from its own trajectories and commits a candidate only if it does not regress on a disjoint held-out split, using a strict keep-better gate. Across four diverse benchmarks, ALFWorld, GAIA, (τ)-bench, and WebShop, and six faithful baselines, ReAct, Reflexion, GEPA, AWM, ACE, and Dynamic Cheatsheet, all evaluated on one shared local backbone, we find three main results. First, no artifact universally wins. RSEA is the strongest single-pass method on ALFWorld, reaching 69.3% compared with 64.6% for ReAct (McNemar (p=0.015)), and reaches 79.4% with retry, the best overall result. However, concrete-workflow induction, represented by AWM, is best on the strong-backbone tool-use tasks. Second, unguarded context evolution is high-variance and unsafe. Dynamic Cheatsheet, which curates context online without a held-out gate, is near-best on ALFWorld at 70.7%, yet collapses on WebShop, with a score of 0.14 compared with 0.43 for ReAct. Third, RSEA's strict held-out selection is what makes recursive self-evolution monotone-safe: it never significantly underperforms the base agent on any benchmark and falls back to vanilla ReAct when evolved context would hurt.
표(tabular) 파운데이션 모델(TFM)은 표준 분류와 회귀(regression)에서 빠른 진전을 이루었지만, 시간-사건(time-to-event) 생존 예측 과제는 대부분 다루어지지 않은 채 남아 있다. 표준 회귀 과제와 달리, 생존 예측 모델은 검열(censored) 데이터를 고려해야 한다. 표준 TFM은 검열 데이터를 네이티브하게 처리할 수 없어 편향되고 부정확한 예측을 초래하며, 이로 인해 실제 응용에 부적합하다. 이 근본적 한계를 극복하기 위해, 우리는 생존 예측 과제를 위한 사전-데이터 적합 네트워크(PFN), \texttt{SurvPFN}을 제안한다. 우리는 검열 데이터를 고려하는 분포 회귀(distributional regression)를 통해 생존을 학습하도록 수백만 개의 합성 생존 예측 과제에서 \texttt{SurvPFN}을 사전학습(pre-training)한다. \texttt{SurvPFN}은 (1) Weibull 사건 시간과 비정보적(non-informative) 검열 메커니즘으로 데이터를 생성하고, (2) 검열된 사건 지표를 통합하며, (3) 검열 음의 로그-가능도(censored negative log-likelihood)를 최소화하는 방식으로 동작한다. 실제 생존 과제 모음인 SurvSet에서 \texttt{SurvPFN}은 데이터셋별 적합, 생존 특화 아키텍처, 또는 특징 공학(feature engineering) 없이도 고전적 및 딥러닝 생존 베이스라인과 매우 경쟁력 있는 성능을 보인다. 우리는 생존을 검열 손실을 갖는 연속 시간 분포 회귀 문제로 취급할 수 있음을 보여주며, 이를 통해 시간-사건 예측에서 PFN의 힘을 활용할 수 있음을 입증한다.
여러 출처의 충돌하는 값에서 올바른 값을 결정하는 데이터 융합(data fusion) 문제에 대규모 언어 모델(LLM)을 활용하는 방법을 탐구하며, 단일 진실·다중 진실 시나리오 모두에서 도메인 의존·독립, 제로샷·원샷 프롬프팅 전략을 세 가지 벤치마크 데이터셋으로 평가한 결과 LLM 기반 접근법이 DART, LTM 등 기존 비지도 진실 발견 방법을 전 데이터셋에서 능가하는 성능을 보였다.
LLM-as-a-Judge 파이프라인이 암묵적으로 가정하는 "평가가 생성보다 쉽다"는 전제를 검증하기 위해, SQuAD 2.0, DROP, HotpotQA, MuSiQue 4개 벤치마크에서 모델이 자신이 생성한 답변을 자기 평가하는 문맥 내 QA 실험을 수행한 결과, 평가가 항상 더 쉽지 않으며 생성 시 어텐션이 컨텍스트에 3~5배 더 집중됨을 발견했다. LoRA 파인튜닝 실험에서도 생성과 평가 능력이 서로 다른 파라미터 구조를 사용함이 확인되어, 자기 평가 파이프라인의 핵심 가정에 의문을 제기한다.
LLM 에이전트가 다중 세션 대화에서 갱신된 사실을 이전 값 대신 현재 값으로 올바르게 유지하는 능력, 즉 supersession 능력이 경계 메모리(bounded memory) 조건에서 심각하게 저하되며(gpt-5.4 기준 92%→77%), 더 큰 모델이나 더 많은 메모리로도 이 격차가 좁혀지지 않음을 실제 대화 데이터로 입증했다. 저자들은 이 측정을 학습 신호로 전환하는 강화학습 환경 Supersede를 공개하고, Qwen2.5-3B를 GRPO로 미세조정하여 held-out supersession 정확도를 9.0%에서 16.7%로 거의 두 배 향상시키는 데 성공했다.
추상화 및 추론 코퍼스(Abstraction and Reasoning Corpus, ARC)에는 제한된 그리드 샘플로부터 패턴을 요약하고 출력 그리드를 예측해야 하는 과제들이 포함되어 있다. 최근 많은 언어 모델 기반 접근법들이 이를 텍스트 기반 추론 과제로 변환하려고 시도해 왔다. 그러나 오픈소스 모델 기반 방법들은 일반적으로 만족스럽지 못한 결과를 보였고, 폐쇄형 모델에 의존하는 방법들은 비용이 너무 크다. 현재의 노력은 주로 데이터 증강, 즉 보다 포괄적인 지도 미세조정을 위한 ARC 유사 데이터 구축에 집중되어 있다. 본 연구에서는 ARC 유사 문제를 해결하기 위해서는 긍정 샘플 지도뿐만 아니라 부정 샘플을 구분함으로써 모델 추론을 개선할 수 있는 능력도 필요하다고 주장한다. 이를 위해 선호 정렬(preference alignment)의 아이디어를 차용하여, 선호 쌍을 구성해 모델이 이를 구분할 수 있게 하는 방법인 DiARC를 제안한다. 구체적으로 부정 샘플을 구성하는 세 가지 방법을 제안하는데, 여기에는 출력 수준의 시각적 변환, DSL 수준의 규칙 역전, 과제별 규칙 편집이 포함된다. 이렇게 생성된 부정 샘플은 관찰된 예시는 변경하지 않은 채 유익한 근접 오답(near-miss) 대안을 제공한다. 여러 ARC 유사 벤치마크에 걸친 실험 결과, DiARC는 기준선 모델 대비 성능을 일관되게 향상시킴을 보여준다. 코드는 https://github.com/szu-tera/DiARC에서 공개되었다.
대규모 언어 모델(LLM)이 의료 환경에 점점 더 배포됨에 따라, 생성되거나 기존에 존재하는 텍스트에서의 정확한 오류 탐지 및 교정은 환자 안전에 위험을 초래할 수 있는 사소한 실수조차 중요해지므로 필수적이 되었다. 오류 탐지 및 교정을 위한 기존 방법들—자동화된 검사와 휴리스틱 기반 접근을 포함하여—은 보지 못한 데이터셋에 대해 잘 일반화되지 못한다. 본 논문에서는 MedGuards를 의료 안전 가드레일로 제안하며, 이는 의료 오류 탐지 및 교정을 다중 에이전트 문맥 내 학습(ICL) 과제로 다루는 새로운 프레임워크이다. 전문화된 에이전트들이 오류를 각각 탐지, 위치 특정, 교정하고, 신뢰도 기반 중재 메커니즘이 추론 추적(reasoning traces)과 신뢰도 점수를 사용하여 의견 불일치를 해결한다. 이 설계는 기본 LLM의 추가 학습을 요구하지 않으면서 해석 가능성, 견고성, 적응성을 향상시킨다. 또한, 참조 텍스트 내 핵심 키워드가 올바르게 생성되었는지를 고려하는 새로운 평가 지표인 Keyword-Prioritized Correction Score(KPCS)를 도입하여, 기존 지표보다 더 포괄적인 평가를 제공한다. 임상 기록으로 구성된 4개의 다국어 의료 데이터셋에 걸친 실험은 제안된 프레임워크가 여러 지표와 모델에서 유의미한 개선을 보여줌을 입증한다. 우리의 목표는 실제 의료 응용에서 LLM의 더 안전한 배포를 가능하게 하는 것이다. 재현성을 위해 코드를 https://github.com/congboma/MedGuards 에 공개한다.