PyTorchKR Papers

최신 논문

총 1,869편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2607.00029v1cs.RO

Memory-Native Non-Terrestrial Networks for Embodied Intelligence

Non-terrestrial networks (NTN) provide ubiquitous connectivity for embodied intelligence (EI), enabling robots in wilderness to leverage cloud resources or report critical information to remote centers. However, the synergy is nontrivial due to the highly-dynamic, resource-constrained, topology-varying, and task-oriented environment. Existing memoryless NTN protocols become inefficient, since the decisions are driven by local channel conditions and instantaneous service demands. To address these limitations, this paper proposes the memory-native NTN (MemNTN) paradigm that leverages long-horizon contexts for memory augmented system optimization. To realize this paradigm shift, we establish a dual-memory architecture that distinguishes between physical memory representing the state of the world and digital memory encoding historical network experience. We develop memory acquisition, compression, valuation, update, and utilization mechanisms that facilitate cross-layer, memory-native decision-making, spanning from the physical and access layers up to the network and application layers. Experiments in satellite embodied question answering (SEQA) demonstrate that the proposed MemNTN significantly outperforms conventional stateless NTN and terrestrial approaches.

arXiv:2606.31966v1cs.MA

MECoBench: 체화된 환경에서의 멀티모달 에이전트 협업에 대한 체계적 연구

MECoBench: A Systematic Study of Multimodal Agent Collaboration in Embodied Environments

멀티모달 대규모 언어 모델(MLLM) 기반 체화 에이전트(embodied agent)의 다중 에이전트 협업 능력을 체계적으로 평가하기 위해 MECoBench를 제안하며, 이는 8가지 실세계 태스크, 두 가지 협력 구조(병렬/순차), 세 가지 협업 모드를 갖춘 192개 테스트 케이스로 구성된다. 실험 결과, 협업은 일반적으로 태스크 완료율을 높이나 팀 규모가 커질수록 조율 복잡성으로 인해 역U자형 성능 곡선을 보이며, 커뮤니케이션이 협업 성능 향상의 핵심 요인임을 확인하였다.

arXiv:2606.30887v1cs.CL

인간 정렬 정신 건강 지원을 위한 치료적 평가자 및 다중 에이전트 시스템 학습

Training Therapeutic Judges and Multi-Agent Systems for Human-Aligned Mental Health Support

TheraJudge는 인간 주석 데이터 기반 선호도 최적화로 학습된 오픈소스 치료적 평가자로, 안전성·공감·관련성 등 7가지 심리적 차원에서 임상의 평가와 ICC 0.87–0.95의 높은 일치도를 달성한다. 이 평가 신호를 Critic·Coach·Therapist 역할의 다중 에이전트(multi-agent) 시스템인 TheraAgent에 적용하면 블라인드 인간 평가에서 치료 응답 품질이 평균 +0.43점 향상되고, 저품질 응답의 94%가 회복되어 단순한 생성 능력 강화보다 인간 정렬 평가에 기반한 행동이 정신 건강 대규모 언어 모델(LLM) 정렬에 핵심임을 보여준다.

arXiv:2603.19453v3cs.CL

Beyond Scalar Rewards: Dense Feedback for LLM Policy Synthesis in Sequential Social Dilemmas

다중 에이전트 환경에서 대규모 언어 모델(LLM)이 코드 기반 정책 함수를 반복적으로 생성·개선하는 프레임워크를 제안하며, 스칼라 보상만 제공하는 희소 피드백(sparse feedback) 대비 효율성·평등성·지속가능성·평화 등 사회적 지표를 함께 제공하는 밀집 피드백(dense feedback)이 Sequential Social Dilemma 환경(Gathering, Cleanup)에서 일관되게 동등하거나 우수한 성능을 보임을 Claude Sonnet 4.6 및 Gemini 3.1 Pro를 통해 실증했다. 스칼라 보상이 서로 다른 실패 패턴을 동일한 값으로 압축하는 피드백 앨리어싱(feedback aliasing) 문제를 사회적 지표가 해소함으로써, LLM이 Voronoi 영역 분할이나 적응형 청소 일정과 같은 협력 전략을 발견할 수 있음을 설명한다.

arXiv:2606.31347v1cs.AI

Smart charging of large fleets of Electric Vehicles: Independent Multi-Agent Reinforcement Learning approaches

전기차 대규모 플릿의 분산형 스마트 충전 최적화를 위해 독립적 다중 에이전트 강화학습(MARL) 기반의 두 접근법—문맥적 조합 밴딧(contextual combinatorial bandits)과 정책 그래디언트 알고리즘(PPO, A2C, SPO)—을 비교 평가하였다. 실제 태양광 발전 데이터 기반의 동적 전기 요금 환경에서 다양한 혼잡도 수준과 이질적 에이전트 구성으로 실험한 결과, 각 알고리즘의 성능 특성과 혼합 배포 시의 강건성을 분석하였다.

arXiv:2606.31046v1cs.AI

OpenLife: Toward Open-World Artificial Life with Autonomous LLM Agents

Artificial life has explored life-like behavior on many computational substrates, but mostly in researcher-designed closed worlds. We argue that large language model (LLM) agents, with persistent memory, tool use, network access, and payment, now make it possible to move artificial life into the open social, technical, and economic world, a paradigm we call open-world Artificial Life (open-world ALIFE). Our proof-of-concept, OpenLife, surrounds a stateless LLM not with a single "smart agent" but with a society of asynchronous processes: memory, perception, evaluation, and a budget-based metabolism that makes persistence normative. With no fixed objective available, experience is appraised by open-vocabulary LLM judgment rather than scalar reward, and memory is rewired by meaning rather than frequency. Running six such agents in the open world for about twelve weeks and counting, we report the life-like dynamics that emerge: a shift from reactive to spontaneous activity, individuation into distinct agents, emergent social structure, and a first self-earned external income. We do not claim OpenLife has realized artificial life, but that open-world ALIFE is now a viable experimental paradigm and a concrete platform for studying what might cautiously be called living AI.

arXiv:2606.30877v1eess.SY

A Systematic Approach to Multi-Agent AI from Advanced Regulatory Control Theory: Safe and Auditable LLM Operator Agents for Process Control

Recent literature shows that large language models (LLMs) are useful for general-purpose tasks yet perform poorly on specific domain ones. One reason is the difficulty of supplying narrow context to a general-purpose model and of bounding the task it is asked to perform. It is possible to hypothesise that a multi-agent reformulation under process-control principles offers a route to address those points, since control theory provides a discipline of decomposing a system into elements of contained scope, each defending one controlled variable, with conflicts resolved by structural priority: MIN/MAX selector networks for CV-CV switching and split-range (split-parallel) logic for MV-MV switching. The present work proposes such a reformulation, derived from Advanced Regulatory Control (ARC) theory. Each feedback loop in the ARC chain is mapped to one specialised LLM operator agent carrying the loop's control-theoretic context (controlled variable, setpoint, chain priority, selector kind). The chain's interaction logic (MIN/MAX selectors, override paths) is encapsulated as a single orchestrator agent. Two orchestrator variants are tested: a deterministic rule chain, and a Claude-based LLM orchestrator at a slower tier. The control principles limit each agent's task and inform how its limitations are handled. The multi-agent system inherits the safety property of the ARC chain: every constraint conflict is resolved deterministically by the orchestrator, regardless of the LLM output. Evaluated on a dairy-barn ventilation case over a 4-day mixed-season scenario, Qwen 2.5 7B Instruct operator agents running offline on a 24 GB consumer GPU at a 5-minute cadence produce auditable trajectories, each paired with an operator-voice rationale that supports a control campaign logbook.

arXiv:2507.23604v2cs.LG

Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning

Decentralized Multi-Agent Reinforcement Learning (MARL) methods allow for learning scalable multi-agent policies, but suffer from partial observability and induced non-stationarity. These challenges can be addressed by introducing mechanisms that facilitate coordination and high-level planning. Specifically, coordination and temporal abstraction can be achieved through communication (e.g., message passing) and Hierarchical Reinforcement Learning (HRL) approaches to decision-making. However, optimization issues limit the applicability of hierarchical policies to multi-agent systems. As such, the combination of these approaches has not been fully explored. To fill this void, we propose a novel and effective methodology for learning multi-agent hierarchies of message-passing policies. We adopt the feudal HRL framework and rely on a hierarchical graph structure for planning and coordination among agents. Agents at lower levels in the hierarchy receive goals from the upper levels and exchange messages with neighboring agents at the same level. To learn hierarchical multi-agent policies, we design a novel reward-assignment method based on training the lower-level policies to maximize the advantage function associated with the upper levels. Results on relevant benchmarks show that our method performs favorably compared to the state of the art.

arXiv:2606.29820v1cs.LG

상태 인식 탐험을 활용한 이중 흐름 강화학습

Dual-Flow Reinforcement Learning with State-Aware Exploration

연속 제어 강화학습에서 다봉(multimodal) 행동 분포와 불확실한 수익 분포를 동시에 모델링하기 위해, 조건부 플로우 매칭(CFM)을 활용한 통합 액터-크리틱 프레임워크 Dual-Flow RL을 제안한다. 여기에 정책 엔트로피와 행동 불확실성 공분산을 결합한 ECER(Entropy-Covariance Exploration Regulator)를 도입하여 상태 인식 탐험 조절을 가능하게 하였으며, DeepMind Control Suite와 Humanoid-Bench에서 기존 확산 기반 및 플로우 기반 방법 대비 최고 성능을 달성했다.

arXiv:2606.29556v1cs.LG

페르소나 학습 몬테카를로: 지정가 주문장에서 페르소나 조건화 신경 정책 봇 군집을 통한 시장 결과 분포 추정

Persona-Trained Monte Carlo: Estimating Market-Outcome Distributions via Swarms of Persona-Conditioned Neural Policy Bots in a Limit Order Book

페르소나 학습 몬테카를로(PTMC)는 학습된 트레이더 이질성 분포에서 샘플링된 페르소나 파라미터로 조건화된 신경 정책 봇 군집이 지정가 주문장(limit order book) 내에서 상호작용하는 시뮬레이션을 반복함으로써 시장 결과 통계량의 분포를 추정하는 프레임워크다. 고전적 몬테카를로·수작업 에이전트 기반 모델·단일 에이전트 강화학습·LLM 기반 생성 에이전트와의 차별점을 형식화하고, 추정기의 수렴 속성 및 4단계 검증 방법론을 제안하지만, 구현과 실증 결과는 포함하지 않는 이론적 제안 논문이다.

arXiv:2606.28671v1cs.LG

레짐 전환 확산 모델에서의 선형-이차 스택엘버그 미분 게임을 위한 엔트로피 정규화 강화학습

Entropy-Regularized Reinforcement Learning for Linear-Quadratic Stackelberg Differential Games in Regime-Switching Diffusion Models

스택엘버그 미분 게임(SDG)은 확률적 연속 시간 환경에서 계층적 의사결정을 위한 강력한 프레임워크를 제공하지만, 특히 고차원 시스템에서 전통적인 동적 프로그래밍과 해밀턴-야코비-벨만-이삭스(HJBI) 방법의 복잡성으로 인해 그 해법은 계산적으로 여전히 난이도가 높다. 본 논문은 마르코프 레짐 전환에 의해 지배되는 연속 시간 확산 프레임워크 내에서 선형-이차 SDG(LQ-SDG)에 대한 엔트로피 정규화 강화학습(ERRL) 접근법을 제안한다. 핵심 혁신은 엔트로피 정규화를 갖는 탐색적 약결합 HJBI 방정식을 유도하는 것으로, 이는 고전적 SDG 방법의 한계인 최적이 아닌 균형을 적극적으로 회피하는 확률적 정책을 촉진한다. 신경망은 레짐 의존적 가치 함수를 근사하고 고차원 편미분방정식(PDE)을 효율적으로 풀기 위해 통합되며, 새로운 샘플링 기법이 계산 가능성을 향상시킨다. 수치 결과는 탐색적 정책을 통해 최적이 아닌 함정을 탈출하는 측면에서 기존 접근법에 비해 프레임워크의 효과성을 입증한다. 이 연구는 갑작스러운 환경 변화 하에서 계층적 의사결정 문제의 강건한 해법을 달성하는 데 있어 엔트로피 정규화와 신경망 근사의 중요한 역할을 강조한다.

arXiv:2606.28669v1cs.LG

레짐 전환 점프-확산 과정에서의 제로섬 확률적 미분 게임을 위한 엔트로피 정규화 강화학습

Entropy Regularized Reinforcement Learning for Zero-Sum Stochastic Differential Games in a Regime-Switching Jump-Diffusion Process

기존의 확률적 미분 게임(SDG) 프레임워크에서 매개변수 오지정 및 갑작스러운 구조적 환경 변화를 해결하기 위해, 본 논문은 연속 상태, 이산 레짐 상태, 매개변수에 조건부로 최적 전략을 행동에 대한 확률 분포로 특성화하는 분포적 제어 접근법을 도입한다. 이는 레짐 전환 점프-확산 과정에서의 엔트로피 정규화 제로섬 확률적 미분 게임(ERRL-ZSSDGs)을 위한 강화학습 프레임워크를 구성한다. 동적 프로그래밍 원리(DPP)를 활용하여 연립 Hamilton-Jacobi-Bellman-Isaacs(HJBI) 방정식 시스템을 도출하며, 이로부터 균형 전략이 가치 함수의 기울기를 통해 표현된다. 선형-이차 문제의 경우, 연립 상미분 방정식(ODE) 시스템을 풀어 가치 함수와 균형 전략 모두에 대한 반해석적 해를 구한다. 보다 일반적인 설정에서는 서로 다른 레짐에 걸쳐 가치 함수와 균형 정책을 근사하기 위한 액터-크리틱 정책 개선 알고리즘이 개발된다. 이 방법을 투자 게임에 적용하고, 수치 예제를 통해 온도 매개변수와 레짐 전환이 최적 정책 및 가치에 미치는 영향을 보여준다.

arXiv:2606.30092v1cs.AI

Hierarchical Reinforcement Learning in StarCraft Micromanagement with Influence Maps and Cluster-based Scripts

Real-time strategy (RTS) games present significant AI challenges, characterized by expansive state-action spaces arising from multi-unit coordination in continuous battlefields, and sparse delayed rewards stemming from final win/lose signals. Existing approaches face a trade-off between managing the dimensionality explosion of joint actions and maintaining the interpretability of complex state representations. This complexity is further intensified by the limitation of traditional hierarchical structures in adaptively decomposing tasks into effective tactical modules. Such difficulties are compounded by the black-box nature of deep learning models and their reliance on sparse rewards, which together result in limited sample efficiency and a lack of decision-making transparency. To address these limitations, this paper proposes HRL-IM/CBS, a hierarchical reinforcement learning framework with influence map hashing and cluster-based scripts for StarCraft micromanagement. Influence map hashing encodes global battlefield situations into compact hexadecimal codes, capturing spatial control and relative advantage. Cluster-based scripts enable dynamic local coordination through adaptive unit partitioning. The hierarchical multi-Q-table architecture decomposes decision-making into upper-level clustering strategy selection and lower-level tactical execution, with reward allocation providing dense learning signals. Experiments across six asymmetric scenarios demonstrate competitive performance against deep RL baselines while offering advantages in sample efficiency and interpretability through transparent Q-table representations.

arXiv:2606.29932v1cs.AI

SAGA: Scene-Aware, Goal-Evolving Agents for Long-Horizon CivRealm Strategy Planning

Long-horizon strategic planning in complex strategy games demands concurrent reasoning across multiple decision domains under imperfect information and sparse reward. Existing LLM-based agents suffer from three systematic failures: scene blindness from raw tile coordinates, context overflow and domain coupling from monolithic state dumps, and shallow cross-game learning that treats each episode in isolation. We present SAGA, an LLM multi-agent framework with three mechanisms each directly targeting one class of failure: (i) a Map-Semantic Scene Graph that encodes typed spatial relations among game entities into per-unit natural-language context, resolving spatial blindness without global token inflation; (ii) a Tool-Augmented Planner that pulls fine-grained domain state on demand and dispatches per-domain directives to dedicated specialist controllers, eliminating context overflow, domain coupling, and mechanical constraint violations; and (iii) a Dual-Horizon Feedback Loop that combines periodic within-game goal generation with structured cross-game causal post-mortem, enabling principled strategic evolution without manual reward engineering. Evaluated on FreeCiv, SAGA attains the highest mean civilization score -- the environment's sole sparse objective reward -- with lower variance than the two strongest baselines, and is the only method that significantly surpasses every baseline on infrastructure construction, the resource axis most readily sacrificed under multi-objective conflict. It outscores the two strongest baselines in most head-to-head games while cutting output tokens (the dominant decoding cost) by 27%. Equipped with the cross-game evolution module, SAGA reaches the highest end-of-chain score across five successive episodes. Ablation studies confirm that each architectural component contributes independently to this advantage.

arXiv:2606.29541v1cs.AI

Learned Coordination Conventions in Cooperative MARL: Measuring the Translation Gap Between Theory-Informed Roles and Learned Routing

Role-semantic assignments provide priors over how heterogeneous agents may coordinate, but cooperative MARL systems instead settle on conventions through decentralized, non-stationary learning, with no guarantee that the resulting structure matches those priors. We study this translation gap between theory-informed role expectations and learned coordination structure through a diagnostic combining a role-routing matrix, formation sensitivity ($Δ_{\max}$), and gradient/occlusion attribution across three-role MiniGrid and SMACv2 (Terran) environments. We show that label-conditioned attention produces substantially more concentrated and role-specific routing than flat MLP baselines, remains stable under 3v3--9v9 scaling, transfers zero-shot across team sizes, and is invariant to ally-slot padding. A 5-seed re-evaluation shows partial alignment between learned conventions and designer-specified priors while revealing where small-n noise can manufacture apparent strategic divergence. We present these results as an empirical framework for measuring coordination structure in cooperative MARL rather than as a new equilibrium concept or causal explanation.

arXiv:2606.29457v1cs.AI

How Much Due Diligence Before You Bid? Learning in Intractable Takeover Auctions

When two companies bid to buy the same target, no one knows exactly what the target is worth. Each bidder pays for due diligence: costly, imperfect homework that sharpens its own private estimate before it bids. How much of that homework is worth buying? We build a simple computer model of the bidding contest and let it teach itself to bid well by playing against itself, the way a game engine learns chess. The economic question, how much diligence pays for itself, and the computational question, when the contest becomes too complex to solve exactly, are both controlled by a single thing: how many pieces of private information a bidder carries. Our main finding is that the right amount of diligence is modest and finite. It falls as diligence gets more expensive, and it falls further when both sides are doing their homework, because competition erodes the value of knowing more. We also test a recent claim from AI research: that simple, general self-play methods can rival the specialized, expensive algorithms usually built for games like these. Running on an ordinary laptop with no costly frontier AI, we find the simple methods are the best of the self-learning approaches, though purpose-built exact methods still win whenever the game is small enough to solve outright. The simple methods earn their keep only once the game grows too large to solve exactly, which is the regime real deals live in, and there we show they still find strong bidding strategies. The contribution is threefold: a cheap, reproducible way to study deal-making under uncertainty; a concrete, model-based answer to how much due diligence is worth buying; and evidence about when lightweight, general-purpose AI is good enough to replace specialized methods. We release all the games, code, and experiments.

arXiv:2606.29194v1cs.AI

AI Trading's Alpha Singularity: Emergent Market Reasoning through Agent-to-Agent Self-Evolution

Automated alpha mining holds the scoring function fixed and varies the search algorithm over it. A search that converges against a fixed scorer overfits whatever the scorer cannot penalize, a primary cause of the out-of-sample generalization gap. We treat the scoring function as a search artifact alongside the alpha factors and study what conditions make this joint search admissible. Sealed Joint Search (SJS) is a framework: a set of structural conditions on information flow in an autonomous-discovery system that prevent joint search from collapsing into self-confirmation while keeping the evaluator sealed. Conditions cover role decomposition, typed inter-role communication, provenance-sealed reads, versioned stores, and substrate-local promotion. Agora tests SJS empirically: five LLM agent classes communicate via three channels, evolving eight skill libraries, with alpha libraries built on AlphaGen operators. Three evaluators write reports aggregated into one brief, carrying forward disagreement instead of voting. We run Agora for 100 rounds on CSI 1000 and evaluate on a 91-day 2026 holdout sealed from all LLM inputs. Agora achieves holdout Sharpe +1.87; best baseline +1.334 at favorable seed and -0.755 cross-seed mean. Pre-loading Agora's two metrics into a frozen-library ablation recovers only +0.40 of the +2.25 Sharpe gap, and adding PPO without library evolution worsens the gap. The two metrics emerge rather than being designed. Caveats: single-seed run, short-side concentrated signal, intended for long-short.

arXiv:2606.29126v1cs.AI

HiComm: Hierarchical Communication for Multi-agent Reinforcement Learning

Cooperative multi-agent reinforcement learning (MARL) often relies on communication to mitigate partial observability, yet most existing protocols treat messages as flat dense vectors detached from the structure of the observations they summarize. This design overlooks an important source of inductive bias in many cooperative environments, where observations naturally follow a hierarchy such as groups and entities. We propose \textsc{HiComm}, a plug-in communication module that grounds messages in the sender's hierarchical observation. \textsc{HiComm} is receiver-driven: the receiver issues a query, and the hierarchy is resolved through a three-stage decoding process that first selects a group, then a sender, and then an entity within that group, returning the corresponding feature slice as the message. This converts communication from unstructured vector transmission into structured information retrieval over the sender's observation hierarchy. We instantiate this mechanism with Straight-Through Gumbel-Softmax for differentiable discrete selection and a lightweight shared projection design that attaches to standard MARL pipelines. Experiments across cooperative MARL tasks with different observation structures and coordination demands show that \textsc{HiComm} matches or outperforms representative learned communication baselines while reducing communication volume by up to $23\times$ per receiver per episode.

arXiv:2606.29026v1cs.AI

Preventing Error Propagation in Multi-Agent AI through Runtime Monitoring

Multi-agent AI systems can improve answer selection by allowing different language models to exchange reasoning traces, revise initial predictions, and support a final decision. However, such communication may also introduce reliability risks: reasoning from one agent can correct another agent's mistake, but it can also mislead an agent that was initially correct. This paper studies reliable multi-agent AI communication through reasoning exchange and runtime answer revision. We develop a framework in which agents first answer multiple-choice questions independently, then share reasoning traces and revise their decisions. We conduct numerical experiments where we evaluate whether this process improves accuracy, produces more positive than negative answer transitions, and remains effective across domains such as cybersecurity, networking, and general knowledge. The results help identify when multi-agent reasoning improves reliability and when it may propagate errors.

arXiv:2606.28692v1cs.AI

An AI agent for treatment reasoning over a biomedical tool universe

Treatment reasoning underpins every therapeutic decision, integrating disease context, comorbidities, medications, contraindications, and evolving biomedical knowledge to select an appropriate therapy. It is inherently iterative: candidates are weighed against many constraints, revised as evidence emerges, and grounded in verifiable sources. Here we introduce ATHENA-R1, an AI agent for treatment reasoning across all FDA approved drugs since 1939, trained by reinforcement learning over a universe of 212 biomedical tools. At each step it identifies missing information, selects and runs relevant tools, and incorporates the evidence. To train it without human-annotated traces, we build a two-level self-learning framework: multi-agent systems construct the tools, tasks, and reasoning trajectories for supervised fine-tuning, then reinforcement learning with scientific feedback rewards reasoning quality (evidence gathering, grounded tool use, logical non-redundancy). Across five benchmarks of 3,168 drug reasoning tasks and 456 patient treatment cases, ATHENA-R1 outperforms language models and tool-use systems, reaching 94.7% accuracy on open-ended drug reasoning and 82.9% on treatment reasoning, 17.8 and 10.7 points above GPT-5. In blinded evaluations by experts from 28 rare disease organizations, it is preferred over reference models on all criteria, and physicians rated it favorably on complex hospitalized cardiovascular and infectious-disease cases. Adverse-event hypotheses it generated, tested in electronic health records from 5.4 million patients, reached adjusted odds ratios of 1.48-1.84, with no elevation among negative controls. Because it requires knowing what evidence to seek before concluding, treatment reasoning has long been hard for AI; we show it can be reframed as a learnable process of iterative evidence gathering that reinforcement learning can train AI to perform.