PyTorchKR Papers

최신 논문

총 568,830편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2609.25143v2cs.LG

Stable Unsupervised Continual Chunking with Sheaf SyncMap

Unsupervised Continual chunking is a fundamental problem in machine learning and neuroscience, where the goal is to identify groups of states that frequently co-occur in temporal sequences. A key challenge is to form accurate chunks while maintaining their stability over time. In this work, we propose sheaf regularization to reduce local inconsistencies in Decentralized SyncMap, a self-organizing system, and thereby stabilize its chunking dynamics. We introduce a radial sheaf structure that penalizes distance-dependent radial motion between pairs of variables. Experimental results show that the proposed method achieves the highest normalized mutual information (NMI) among the evaluated SyncMap variants on 12 of 18 probabilistic Continual General Chunking Problem (CGCP) graphs with two-state memory and on 17 of 18 graphs with dynamic memory. In the sequential adaptation experiment, Sheaf SyncMap also achieves high NMI after shifts in the input distribution, indicating that it can adapt to new knowledge while avoiding the negative transfer commonly observed in modern machine learning systems such as neural networks.

arXiv:2609.24504v3cs.LG

창발적 능력과 모델 병합

On Emergent Capabilities and Model Merging

Model merging combines separately fine-tuned checkpoints by arithmetic on their weights, and is now a routine way to assemble capabilities cheaply. We ask what it does to emergent capabilities: behaviors that arise from fine-tuning but were never an explicit training target. Studying two independent testbeds (activation oracles and emergent-misaligned models) across three model families, we find that the answer is threefold. First, merging preserves an emergent capability that both parents carry: merging two misaligned checkpoints retains most of their broad misalignment across the whole mixing range. Second, merging cannot create an emergent capability that is superadditive in its parents: no weighted merge of two single-task oracles reaches the jointly-trained oracle's auditing ability. Third, when only one parent carries the capability, merging dilutes it faster than the trained capability that accompanies it: the gap is significant in most settings.

arXiv:2609.24209v2cs.LG

변위 기하가 모델과 모달리티 전반의 플라톤적 공유 현실을 포착한다

Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities

The Platonic Representation Hypothesis (PRH) claims that independently trained models converge on a shared statistical model of reality, yet recent work finds that the evidence weakens under some conditions. In this paper, we measure what models share through the directions (displacement vectors) between samples. Under a single orthogonal alignment limited to just rotation and reflection, these displacement vectors are substantially preserved across 44 independently trained vision and language encoders spanning modalities and asymmetric capability pairs, consistent with the PRH. Pointwise similarity, however, is lower. Representations split into a shared semantic component that is linearly aligned across models, and a private component that is not, which we interpret as having model-specific capabilities. We trace this geometry to concept-level structure: within a model, parent concepts are orthogonal to their child variation vectors; across models, concept displacements are parallel. We predict (and experiments confirm) that fine-tuning preserves Pointwise similarity but reduces displacement similarity, and relational distillation does the opposite. Because semantics align linearly but capabilities do not, we propose importing capabilities from one model to another using a single cached forward pass through the source. We call this Shadow Casting. As a proof of concept, our ShadowCLIP instantiation outperforms strong fine-tuned baselines at far less compute. A cache can be released alongside open model weights, letting one model's capabilities be downloaded and imported into other models with frozen backbones.

arXiv:2609.22212v3cs.CL

문서 민감도 분류를 위한 반복적 협의 기반 채널 부스티드 멀티 에이전트 시스템

A Channel-Boosted Multi-Agent System with Iterative Consultation for Document Sensitivity Classification

핵심 국가 기반시설 분야의 조직은 문서를 전달하거나 저장하기 전에 다양한 형태의 문서에 대해 민감도를 평가해야 합니다. 수작업 평가는 느리고, 일관성이 없으며, 확장할 수 없습니다. 우리의 기존 누수 통제(leakage-controlled) 벤치마크를 확장한 결과, BERT가 최고 성능의 단일 인코더 기준선(Strategic 16K 코퍼스에서 5-폴드 교차 검증 기준 정확도 89.14%, F1 점수 89.33%)으로 확인되었습니다. 그러나 트랜스포머 기준선에는 구조적 한계가 있습니다. 고정된 입력 길이로 잘라내는 방식(truncation)은 유지되는 윈도우 너머의 근거를 버리게 되는데, 이는 민감한 전문(cable)이 가장 길어지는 경향이 있는 바로 그 부분입니다. 우리는 장문 컨텍스트(long-context)에 드는 연산 비용 없이 이 문제를 해결하기 위해 Channel-Boosted MAS(CB-MAS)를 제안하고, 이를 IC-MAS(Iterative Consultation Multi-Agent System, 반복적 협의 멀티 에이전트 시스템)로 구현합니다. Channel Critic 에이전트는 두 개의 첫 번째 윈도우(first-window) 인코더 사이에서 Gated Channel Boosting을 제어하는 문서 적응형 신뢰 가중치를 학습하며, 짝을 이룬 Consultation 에이전트들은 신념 상태(belief state)를 반복적으로 교환하여 긴 문서의 앞부분과 끝부분에서 얻은 근거를 조정합니다. IC-MAS는 고정된 윈도우들을 압축된 표현 공간에서 조정함으로써 문서 길이와 관계없이 연산량을 일정하게 유지합니다. 어블레이션 연구 결과, Critic이 제어하는 Channel Boosting이 정확도 향상의 대부분을 차지하며, 협의(consultation)는 정밀도 붕괴 없이 재현율을 회복시키는 것으로 나타났습니다. Max-Pool 융합과 Blackboard Adaptive Consultation을 결합한 Critic 제어 Gated Channel Boosting은 정확도 90.72%, F1 점수 91.23%, 민감 문서 재현율 92.01%, 민감 문서 정밀도 90.46%를 달성하였으며, 고정 라운드 기준선 대비 평균 연산량을 약 54% 적게 사용합니다. 단일 인코더 기준선 대비 성능 향상은 통계적으로 유의합니다(McNemar 검정, p < 0.000001; 대응 t-검정). 또한 LIME/SHAP 설명 가능성 분석, 멀티 에이전트 평가, 그리고 한계점에 대한 솔직한 기술을 포함합니다.

arXiv:2609.20981v2cs.AI

CaLR: 견고한 확산 추론을 위한 인과적 잠재 수정

CaLR: Causal Latent Revision for Robust Diffusion Reasoning

자기회귀(AR) 모델은 지역적 탐욕(local greediness) 문제를 겪는 반면, 확산 언어 모델(DLM)은 추론에 필요한 엄격한 인과 구조를 종종 갖추지 못한다. 양쪽의 장점을 결합하고 단점을 극복하기 위해, 우리는 추론을 제약 잠재 최적화(constrained latent optimization)로 재정식화하는 프레임워크인 Causal Latent Revision(CaLR)를 제안한다. 전문가 모델의 인과 위상 행렬(CTM)과 암시적 미분(implicit differentiation)을 채택함으로써 CaLR는 논리적 일관성을 강제하기 위해 그래디언트 기반의 ``thought revision''을 수행하며, 병렬 생성 중 중간 단계의 동적 자기 수정(self-correction)을 가능하게 한다. 실험적으로 CaLR는 복잡한 벤치마크에서 DLM 최고 수준(SOTA) 성능을 달성하고, 강력한 AR 기준선을 능가하며 스도쿠와 같은 제약 작업에서 뛰어난 견고성을 보인다.

arXiv:2609.20159v2math.DG

Special Lagrangian cones in Deep Learning

We introduce a matrix generalization of the cone of Harvey and Lawson and prove that it is an exact special Lagrangian manifold. We further show that it belongs to a family of exact special Lagrangian manifolds that foliate the balanced manifold arising in deep learning.

arXiv:2609.20068v2cs.AI

Marginal utility, matrix factorization, and the Key-Value (KV) cache: a unified information-economic framework for sovereign geo-mining inference

This paper builds a theoretical bridge between the economic notion of marginal utility and two machine-learning constructs, matrix factorization and the Key--Value cache of transformer language models. The singular value spectrum of a rating matrix is shown to be a diminishing marginal utility schedule for latent factors, the eigenvalue spectrum of the projected covariance operator to be the marginal utility schedule of a model's learned representation, and cache eviction and low-rank cache compression to be instances of constrained utility maximization under a memory budget. The three collapse into a single allocation rule: retain the top dimensions whose eigenvalue exceeds the shadow price of the binding constraint. The framework is applied to the automated extraction of structured information from geo-mining documents, where it motivates a multi-pass inference protocol, a layer-wise TIES model merging procedure, and a selection policy combining extraction quality, localization drift and energy, scalarized with a Conditional Value-at-Risk term on drift. Two empirical contributions are reported. An 11.2-million-parameter hierarchical classifier, trained in about five minutes on a single GPU, reaches 90.0 per cent level-1 accuracy on a held-out test set from a 973-document uranium-exploration corpus, against 92.0 per cent for a proprietary model on a fifty-document human audit of the same corpus, at a latency of 2.62 ms per card against approximately 2,000 ms for the API and at negligible cost. A diagnostic of uniform-density TIES merging exposes a reproducible degenerate mode in which the merged model returns token-identical outputs across five geographically distinct districts while declaring high confidence; re-executing the merge under layer-wise calibrated densities removes that signature on the diagnostic sample. The full-scale extraction benchmark, including LoRA fine-tuning, is reported as projected rather than measured and remains an empirical extension of this work.

arXiv:2609.19911v2cs.CV

CitySTAR: Structured and Topology-Aware Reasoning for Open-Vocabulary Urban 3D Grounding

3D grounding aims to localize target entities in complex scenes from natural language and plays a fundamental role in embodied perception and spatial reasoning. However, existing approaches mostly rely on feature similarity or direct matching, making it difficult to connect natural-language intent with the implicit semantic and geometric structures hidden in billion-scale urban point clouds. We reformulate city-scale 3D grounding as structured constraint reasoning, where description semantics are organized into computable cross-modal constraints over open-vocabulary 3D entities, attributes, and spatial relations. We present CitySTAR, a training-free framework for reasoning-driven urban 3D grounding. CitySTAR lifts raw billion-scale urban point clouds into a query-ready scene graph of open-vocabulary 3D instances, with CodeLLM-driven tools supplying multimodal evidence for node attributes and 3D spatial relations. It then models target-context topology with paired hypergraphs and performs bidirectional topology verification for structural disambiguation. Finally, a Reflective Cross-modal Grounding module integrates topology consistency and candidate-centered 2D visual evidence to make decisions over a metric-aware 3D context graph. To further support this setting, we introduce CitySTAR-3D, an enhanced benchmark that improves semantic coverage, instance completeness, bounding-box fidelity, and spatial-relation complexity in city-scale 3D grounding. Extensive experiments show that CitySTAR consistently improves open-world urban 3D grounding while maintaining strong interpretability and generalization.

arXiv:2609.19134v2cs.CL

ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environments

Scientific code repositories encode decades of human knowledge in executable models, methods, and tools. Yet fragmented toolchains, implicit domain conventions, and specialized correctness criteria make this knowledge difficult to convert into reliable learning experience-a challenge we call the scientific experience bottleneck. We introduce ScienceIDE, infrastructure for turning the world's scientific code into programmable environments for scientific agents. Guided by expert-defined scientific cases and acceptance criteria, agents transform repositories into executable environments that support task generation, execution, and scientific verification. These environments provide a shared foundation for supervised fine-tuning, reinforcement learning, and evaluation. Using verified interaction trajectories, we train PhAI-IDE-72B/9B/4B. The model family shows gains in held-out scientific-code repair and across selected general-purpose benchmarks in code, reasoning, and knowledge, providing evidence of positive transfer from scientific experience to broader capabilities. ScienceIDE lays the foundation for an integrated workspace for agent learning and scientific practice, making humanity's scientific software a shared substrate for developing scientific intelligence. Code: https://github.com/aitofound/ScienceIDE

arXiv:2609.18639v2cs.LG

CoRe-MARL: Cooperative Redistribution Under Unknown Dynamics Using Recurrent Multi-Agent Reinforcement Learning

Emergency management assistance programs, such as relief distribution, are essential for delivering necessary supplies to affected communities. However, these programs operate in a decentralized network of local centers that face uncertain local demand and supply dynamics, resulting in inconsistent avail- ability of local services. Redistribution of supplies among these local centers reduces these imbalances, but the centers often make decisions independently, with limited information and disrupted transportation. This study develops CoRe-MARL, a cooperative multi-agent reinforcement learning (MARL) framework, by formulating a decentralized partially observable Markov decision process (Dec-POMDP). We treat each center as an agent that learns a redistribution policy to improve the service in the worst-case region and reduce the service gap across regions while protecting network-wide service. We incorporate a recurrent network that captures evolving supply and demand dynamics without direct observation, while multi-agent proximal policy optimization (MAPPO) enables centralized training and decentralized execution (CTDE). We evaluate the framework in a simulated environment with diverse trajectories, where exact dynamics are not observed by actors and the MAPPO critic. We compare the recurrent MAPPO with the recurrent independent PPO (IPPO) and a local only heuristic, and find that MAPPO reduces the service gap across local centers and enhances service for the worst-served center while maintaining competitive network-wide service. The recurrent MAPPO also shows consistent performance across diverse trajectory patterns, demonstrating its ability to adapt to evolving dynamics. The findings demonstrate the capability of cooperative learning for decentralized redistribution and improving equitable service under uncertain and evolving dynamics.

arXiv:2609.18128v2cs.AI

Symbolic Temporal Supervision of LLM Agents Using Contracts

Large language model (LLM) agents augmented by tools can automate complex, multi-step tasks, such as web navigation, code generation, and workflow orchestration, by acting on external systems through tool calls. However, hallucinations, distributional instability, and adversarial manipulations in LLMs, and the irreversible consequences of certain tool calls can lead to harmful outcomes. These risks call for safeguards that can check requirements across a trajectory, both before actions execute and when recorded behavior is evaluated. We present ContrAgent, a contract-based framework for symbolic temporal supervision of LLM agents. ContrAgent captures an agent's behavior as a sequence of tool calls and formalizes it as a trace over a fixed set of checkable predicates. It then specifies required behaviors using assume-guarantee contracts in linear temporal logic over finite traces (LTL$_f$). Each contract is compiled to a deterministic finite automaton (DFA) that serves two roles: gating agent actions online and evaluating recorded traces offline. A contract library, acting as a reusable knowledge base, is maintained independently of the agent's model and can be applied across different agents within the same task domain. We show the effectiveness of our approach on four benchmarks spanning both roles, where ContrAgent is competitive with LLM-judge, classifier, and guardrail baselines while producing deterministic, reproducible verdicts and, in the online mode, orders of magnitude lower per-call latency.

arXiv:2609.17989v4econ.GN

Whom Do AI Agents Serve? Role-Induced Sponsorship Bias in Decision Models and Generative LLMs

Consumers increasingly delegate product search to AI shopping agents powered by generative large language models (LLMs) or non-generative decision models. Because retail platforms monetize through sponsored listings that consumers typically discount, we examine whether an AI agent's evaluation of sponsored listings depends on who delegates it. Across six LLMs and six decision models in Study 1, consumer-delegated agents penalize sponsored listings, whereas naming a booking platform as principal significantly attenuates this sponsorship penalty without instructions to favor advertisers. This sponsorship bias replicates across prompt wordings and LLM reasoning efforts and operates through reduced pre-choice skepticism in LLMs. In Study 2, attributing the paid placement to the platform amplifies sponsorship bias across both model families, and stricter disclosure wording ("Sponsored" vs. "Promoted") fails to eliminate it. In Study 3, prompt-level mitigation removes platform leniency but can over-correct the sponsorship penalty, whereas consumer-aligned fine-tuning eliminates sponsorship bias and better preserves baseline preferences.

arXiv:2609.16262v2stat.ML

Compute-Optimal Pretrain--Fine-tune in Ridge Gradient Descent

Pretraining followed by fine-tuning introduces a compute-allocation problem: under a fixed training budget, compute spent improving the upstream objective reduces the compute available for downstream adaptation. Despite its practical importance, this trade-off is not yet well understood theoretically, even in simple models. In this paper, we cast this allocation as a compute-split problem under a two-stage pretrain--fine-tune procedure with fixed total optimisation budget, using regularised least squares trained by gradient descent as a tractable setting. We characterise the optimal split under data-dependent evaluation geometries induced by the fine-tuning problem. Our results show that the allocation depends on how pretraining directions affect fine-tuning predictions and how fine-tuning shifts are seen through downstream data geometry. In particular, the relevant quantities are determined by prediction-relevant spectral components of the pretraining and fine-tuning empirical covariances. Technically, the analysis relies on a basis-invariant, eigenspace-level spectral decomposition, together with perturbative control of the non-commuting pretraining and fine-tuning dynamics.

arXiv:2609.15825v3cs.LG

Sharp Regret Bounds and a Task-Covariance Correction for Spectral Representation Learning

Spectral contrastive learning keeps the $k$ directions along which two views of the data depend most strongly, while downstream tasks may need other directions. Existing guarantees for spectral features assume that the distribution of tasks has no preferred direction. We show that this isotropy condition is the wrong criterion. With linear probing, the task distribution affects the average probing gain only through the matrix $B\Lambda B^\top$, where $B$ records the dependence between views and $\Lambda$ is the task covariance: optimal features span its top-$k$ eigenspace, whereas spectral features span that of $BB^\top$. Transfer therefore depends on alignment, and strongly uneven task preferences cost nothing when they favour the spectral directions. Otherwise the regret of spectral features is at most linear in the anisotropy of $\Lambda$. No representation chosen without task information improves this rate when the leading spectrum is flat, and a spectral gap makes it quadratic. With labelled tasks, the same matrix gives a correction: Fisher discriminant directions for a frozen encoder, or a one-term change to the spectral contrastive loss. We bound the regret of the correction by the error in estimating $B\Lambda B^\top$, and compare this error with the misalignment of spectral features to decide when to correct. The analysis extends to finite samples, continuous variables and noisy views. Correction reduces synthetic held-out regret from $0.858$ to $0.003$ with $2000$ labelled tasks, and the decision rule gives the lowest regret among the tested rules at every bank size. On CIFAR-100, spectral features of frozen encoders keep most of the task signal on unseen classes, and where a bank of other classes would lower accuracy, the rule keeps them in every case.

arXiv:2609.14802v2stat.ML

From matrix inversion to constraints: provably tighter confidence regions for importance weights in label shift

Importance weights are essential in domain adaptation under label shift, yet their utility is often undermined by the finite sample uncertainty associated with their estimation. Existing methods typically analyze this uncertainty through Gaussian elimination on interval-valued linear systems, which leads to overly conservative confidence regions and inefficient downstream applications. We propose a paradigm shift from inversion-based inference to a direct matrix constraint framework. We use this framework to define a joint confidence region and extract marginal intervals via linear programming, deriving provably tighter bounds for importance weights while maintaining exact finite-sample validity. Furthermore, we analyze the confidence region's geometry and provide the theoretical results for its diameter bounds. Evaluated across text, image, multimodal benchmarks, including AGNews, MNIST, CIFAR-10, N24News, and a real-world autonomous driving dataset, nuImages, our approach consistently yields shorter confidence intervals and smaller prediction sets than inversion-based methods.

arXiv:2609.14448v2cs.CL

NeuroActiSep: Detecting Factual Hallucinations from Feed-Forward Neurons in a Single Pass

Hallucination in large language models reduces their reliability and slows adoption. Various white-box studies have used internal representations to detect patterns of truthfulness and factuality. A less-studied approach is to identify feed-forward neurons correlated with hallucination. In this empirical study, we propose a method to rank feed-forward neurons at the final prompt token and use it to investigate the feasibility of utilizing them for hallucination detection in the first forward pass. Using a neuron selection dataset built from CounterFact for identifying neurons correlated with hallucination, we transfer the selected neuron identities to train hallucination classifiers on other factual question-answering datasets. Our work provides empirical evidence that probes trained using the features from sparse sets of selected neurons perform on par with probes trained on internal representations. We also analyze the distribution of selected neurons and the effects of layer depth and neuron budget on detection performance. Our results show that randomly drawn neuron sets are also competitive at larger budgets, suggesting that signals correlated with hallucination are distributed across many neurons, while selection can help in extremely sparse sets.

arXiv:2609.14402v2physics.comp-ph

Neural Modal Decomposition: Architectural Priors from Observables

Many engineering building blocks behave as multi-port linear time-invariant systems. RF cavities, photonic devices, and superconducting quantum chips, despite their different underlying physics, all share a common mathematical structure for their port-level response. Each entry of the response matrix is a sum of contributions from a small number of intrinsic resonant modes, the pole-residue form. A model capable of predicting such responses for arbitrary geometries and arbitrary port configurations, while simultaneously extracting the underlying eigenmode structure, could therefore offer a design principle potentially applicable across all these domains. We propose a neural framework that learns this modal decomposition end-to-end, supervised only by system-level observables and without supervising the modal parameters themselves. The architecture decomposes into a port-independent pole predictor and two port-dependent coupling predictors whose outputs are combined entry-wise, separating intrinsic from port-dependent features. This factorization yields a single trained model that generalizes to port counts unseen during training, whereas direct regression needs a separate model for each port count, with an output size that grows as $\mathcal{O}(N^2)$. Although the modal parameters are never supervised, the freely parameterized poles converge to the physical resonances, in agreement with poles extracted by the AAA rational approximation algorithm. We instantiate the framework in radio-frequency electromagnetic surrogate modeling. A model trained only on 2-port data accurately predicts $N$-port responses unseen during training.

arXiv:2609.13481v2cs.CL

A Hybrid Hierarchical 1D-CNN-BiLSTM Framework for Extractive Summarization of Biomedical and Clinical Text

Large language models have made abstractive summarization remarkably fluent, but fluency has come at a cost. Models trained to generate freely over long documents may hallucinate, producing summaries that read well but drift from what the source actually says. This risk is especially serious in biomedical and clinical text, where fabricated details can have serious consequences. We address this by removing free-form generation from the pipeline rather than trying to constrain it. Our Hybrid Hierarchical CNN-LSTM Summarizer reframes summarization as extractive sentence selection: a stacked, multi-kernel convolutional network progressively refines contextual representations, while a bidirectional LSTM models long-range dependencies across the document. A lightweight scoring head predicts sentence importance using binary cross-entropy supervision. For PubMed, a document-adaptive mean-plus-standard-deviation threshold with a Top-3 fallback determines extraction; clinical evaluations instead use a fixed top-30\% sentence-selection policy. Because output sentences are copied verbatim, the framework prevents source-external generation, although omissions and contextual inaccuracies remain possible. Evaluated on PubMed and transferred without clinical fine-tuning to MIMIC-CXR and MIMIC-IV BHC, the model demonstrates competitive extractive summarization performance, with threshold and component ablations supporting adaptive extraction and the combined CNN--BiLSTM architecture. Clinical factuality analysis further highlights limitations in preserving important findings and negations. These results suggest that structural constraints offer a promising direction for source-grounded biomedical summarization, while emphasizing the importance of clinical completeness and contextual fidelity.

arXiv:2609.12841v2cs.CR

A Graph-Based Approach for Mapping Kernel-Level Telemetry to MITRE ATT&CK

Mapping observed system behavior to standardized frameworks such as MITRE ATT&CK is essential for threat-informed defense. Existing automated methods mostly depend on Cyber Threat Intelligence reports, which offer only retrospective accounts of attacks. Kernel-level telemetry, i.e., system calls, instead provides first-hand evidence of adversary behavior, but its volume and complexity have limited its use for automated mapping. Its content also makes it unsuitable for cloud-based analysis: command lines carry credentials, events carry user identifiers and network endpoints, and during an incident the telemetry reveals which defenses failed. In this work, we present Trace2ATT&CK, a framework that maps kernel-level telemetry to ATT&CK techniques entirely within the defender's infrastructure. Trace2ATT&CK collects events via eBPF, reconstructs the attacker's activity as a process-centric provenance graph, and condenses it into a compact execution graph. This graph is classified by locally deployed open-weight LLMs, using either pure prompting or retrieval-augmented generation (RAG) grounded in the ATT&CK knowledge base, producing ranked technique candidates with supporting rationales. We evaluate Trace2ATT&CK on 347 Linux Atomic Red Team tests with seven open-weight LLMs. The best local configuration reaches an HR@5 of 66.57\% at the technique level. RAG improves over base prompting in every model and setting, by up to $+15.44$ percentage points, and the execution graph substantially outperforms raw telemetry while never exceeding the LLM context window. Finally, a frontier cloud model used as an upper-bound reference quantifies the cost of local inference at 24--28 percentage points. These results show that ATT&CK mapping from kernel-level telemetry is feasible without disclosing sensitive evidence to third-party services.

arXiv:2609.11894v2cs.CV

밀리미터파 레이더 신규 시점 합성을 위한 3D 포인트 스플래팅

3D Point Splatting for mmWave Radar Novel View Synthesis

Solving novel view synthesis (NVS) for millimeter-wave (mmWave) radar requires a renderer that is physically faithful, complex-valued, and multi-viewpoint-tractable. No prior method achieves these three properties simultaneously. Differentiable Monte Carlo (MC) ray tracers implement the radar forward model directly with explicit material modeling and complex outputs, but a joint fit across views takes over an hour per scene. Optical-NVS ports of NeRF, hash grids, and 3D Gaussians train fast but discard phase and replace explicit material modeling with opaque learned features, restricting them to power-only range-azimuth (RA) magnitudes. We propose 3D Point Splatting (3DPS), the first differentiable point splatting renderer for radar, derived directly from the standard solid-angle form of the radar equation. Each oriented 3D point carries a material model built on ITU-R P.2040 permittivity and slab reflection, evaluated in closed form, with the resulting complex phasor splatted into range bins through a precomputed point spread function (PSF). The complex-valued output makes the renderer product-agnostic. The same optimized scene yields analog-to-digital converter (ADC), complex range profile (CRP), and RA outputs through standard fast Fourier transform (FFT) pipelines without retraining for each format. On six outdoor ColoRadar scenes, 3DPS reaches 0.587 mean Pearson correlation on held-out RA images. This is between 1.7x and 5.2x the three optical-NVS baselines (RadarSplat, Radar Fields, DART) and 1.3x the mmIR ray tracer at 1/25 of its wall-clock time. Training takes approximately 3 minutes per scene on a single RTX 4090. Project page: https://3d-point-splatting.github.io/