PyTorchKR Papers

최신 논문

총 817편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2603.14094v3stat.ML

Maximin Robust Bayesian Experimental Design

We address the brittleness of Bayesian experimental design under model misspecification by formulating the problem as a max--min game between the experimenter and an adversarial nature subject to information-theoretic constraints. We demonstrate that this approach yields a robust objective governed by Sibson's $\alpha$-mutual information~(MI), which identifies the $\alpha$-tilted posterior as the robust belief update and establishes the R\'enyi divergence as the appropriate measure of conditional information gain. To mitigate the bias and variance of nested Monte Carlo estimators needed to estimate Sibson's $\alpha$-MI, we adopt a PAC-Bayes framework to search over stochastic design policies, yielding rigorous high-probability lower bounds on the robust expected information gain that explicitly control finite-sample error.

arXiv:2602.18958v3stat.ME

Structure-Adaptive Estimation of Heterogeneous Treatment Effects with Kernel Methods

We study conditional average treatment effect (CATE) estimation from observational data in regimes with potentially weak overlap, when the nuisance response functions lie in a reproducing kernel Hilbert space (RKHS), but the contrast function is structurally simpler. We consider three forms of such simplicity: (i) membership in a lower-complexity RKHS with faster eigenvalue decay, (ii) a source condition relative to the nuisance kernel, or (iii) dependence on a known low-dimensional covariate representation. We develop a unified two-stage kernel ridge regression procedure that remains valid under weak overlap and attains minimax rates determined by the complexity of the contrast function. The procedure avoids propensity-score estimation and separate nuisance convergence-rate assumptions. We also show that a simple model-selection step over candidate contrast spaces and regularization levels satisfies an oracle inequality, enabling adaptation to unknown CATE regularity.

arXiv:2602.04042v3cs.LG

Partition Tree: Conditional Density Estimation over General Outcome Spaces

We propose Partition Tree, a tree-based framework for conditional density estimation over general outcome spaces that supports both continuous and categorical variables within a unified formulation. Our approach models conditional distributions as piecewise-constant densities on data-adaptive partitions and learns trees by greedily minimizing a surrogate for conditional log-loss. This yields a scalable alternative to existing probabilistic trees that does not make parametric assumptions about the target distribution. We further introduce Partition Forest, an ensemble extension obtained by averaging conditional densities. Empirically, we demonstrate improved probabilistic prediction over CART-style trees and competitive performance compared to state-of-the-art probabilistic tree methods and Random Forests.

arXiv:2601.02322v3stat.ME

Environment-Adaptive Covariate Selection: Learning When to Use Spurious Correlations for Out-of-Distribution Prediction

A common approach to out-of-distribution prediction restricts models to causal or invariant covariates to avoid spurious associations that may change across environments. Despite its theoretical appeal, this strategy can underperform empirical risk minimization when only a subset of the causal parents of the outcome is observed. In such settings, non-causal covariates can serve as proxies for unobserved causal parents and improve prediction when the proxy relationship is stable, but they can hurt when shifts disrupt that relationship. Thus, the optimal covariate set can depend on the specific shift encountered. Because different shifts can leave distinct signatures in the covariate distribution, we propose an environment-adaptive covariate selection algorithm that chooses which covariates to use in each environment based on summaries of its unlabeled covariates. The summaries may be hand-crafted or learned from multi-environment data, and prior causal knowledge can be incorporated as constraints. We give sufficient conditions under which such summaries are informative in linear causal models, and explain when adapting only the covariate set is preferable to letting every coefficient vary with the environment. In simulations and two applications, adapting the covariate set improves prediction over static baselines, including causal and invariant covariate selection.

arXiv:2510.00569v2stat.ML

Guaranteed Noisy CP Tensor Recovery via Riemannian Optimization on the Segre Manifold

Recovering a low-CP-rank tensor from noisy linear measurements is a central challenge in high-dimensional data analysis, with applications spanning tensor PCA, scalar-on-tensor regression, tensor-on-tensor regression, and beyond. We exploit the intrinsic geometry of rank-one tensors by casting the recovery task as an optimization problem over the Segre manifold, the smooth Riemannian manifold of rank-one tensors. This geometric viewpoint yields two powerful algorithms: Riemannian Gradient Descent (RGD) and Riemannian Gauss-Newton (RGN), each of which preserves feasibility at every iteration. Under mild noise assumptions, we prove that RGD converges at a local linear rate, while RGN exhibits an initial local quadratic phase followed by linear refinement to the statistical error. Extensive synthetic experiments validate these convergence guarantees and demonstrate the practical effectiveness of our methods.

arXiv:2508.00110v3stat.ML

funOCLUST: 이상치를 포함한 함수형 데이터 클러스터링

funOCLUST: Clustering Functional Data with Outliers

함수형 데이터(functional data)는 무한 차원의 특성과 이상치(outlier)에 대한 잠재적 민감성으로 인해 클러스터링에 고유한 어려움을 제기한다. 이러한 문제를 해결하기 위해 OCLUST 알고리즘을 함수형 설정으로 확장하는 방법을 제안한다. 이 접근법은 OCLUST 프레임워크를 활용하여 곡선을 클러스터링하고 이상치를 제거하는 강건한(robust) 방법을 구현한다. 제안된 방법론은 시뮬레이션 및 실세계 함수형 데이터셋 모두에서 평가되어 클러스터링과 이상치 식별에서 우수한 성능을 입증한다.

arXiv:2505.20536v2stat.ML

Covariate-Adjusted Deep Causal Learning for Heterogeneous Panel Data Models

This paper studies the task of estimating heterogeneous treatment effects in causal panel data models with covariate effects. We propose a novel Covariate-Adjusted Deep Causal Learning (CoDEAL) framework that cohesively deals with the underlying heterogeneity and nonlinearity of both panel units and covariate effects. CoDEAL integrates nonlinear covariate effect components (parameterized by a feed-forward neural network) with nonlinear factor structures (modeled by a multi-output autoencoder) to form a heterogeneous causal panel model. The nonlinear covariate component flexibly captures complex covariate influences on outcomes, and the nonlinear factor decomposition enables CoDEAL to effectively capture both cross-sectional and temporal dependencies inherent in the data panel. This latent structural information is subsequently integrated into a customized matrix completion algorithm, thereby facilitating more accurate counterfactual imputation. Moreover, the use of a multi-output autoencoder explicitly accounts for heterogeneity across units and enhances the model interpretability of the latent factors. We establish theoretical guarantees on the convergence of the estimated counterfactuals and demonstrate the compelling performance of the proposed method using extensive simulation studies and real-world data applications.

arXiv:2605.16757v3cs.AI

NeuroMAS: Multi-Agent Systems as Neural Networks with Joint Reinforcement Learning

Multi-agent language systems are often built as hand-designed workflows, where agents are assigned semantic roles and communication protocols are specified in advance. We propose NeuroMAS, a framework that treats a multi-agent language system as a trainable and scalable neural-network-like architecture with hidden textual intermediate signals. In NeuroMAS, language-agent nodes are role-free but structure-aware: the topology determines how information can flow, while reward-based training determines how nodes communicate, specialize, and coordinate. This formulation shifts multi-agent design from workflow engineering toward architecture design, where depth, width, connectivity, and growth protocol become scalable sources of capability. We provide a theoretical perspective showing why such modular text-valued computation can be parameter-efficient when tasks admit stable decompositions. Experiments show that NeuroMAS improves over both inference-time collaboration methods and trained baselines, and that its gains are not explained by trainable parameter count alone. We further find that organizational scaling is path-dependent: larger systems can fail when trained from scratch, but become stronger when grown progressively from smaller trained systems. These results suggest that learned multi-agent organization is a promising scaling axis for language-model reasoning and generation.

arXiv:2611.09698v1stat.CO

A Generalized Bayesian Approach to Clustering with Dimension Reduction

Generalized Bayes product partition models provide a probabilistic counterpart to loss-based clustering. They update a prior over partitions through an exponentiated clustering loss rather than a likelihood, so allocation uncertainty can be quantified without a full generative model for the data. However, their performance depends on the space in which the loss is evaluated. For high-dimensional observations, a squared Euclidean loss on the raw features can be dominated by irrelevant coordinates or fail to reflect cluster structure lying near a nonlinear low-dimensional manifold. We propose an extension in which an autoencoder and the partition are learned together, so that the space in which the clustering loss is evaluated is shaped by the clustering itself. Our generalized posterior combines a reconstruction loss, which keeps the low-dimensional representation faithful to the observed data, with a within-cluster sum of squares in the latent representation, which encourages cluster separation. Posterior computation alternates between Gibbs sampling over partitions and gradient-based updates of the autoencoder network, with the learning rate estimated jointly. We illustrate the approach through simulation studies and applications to four benchmark datasets.

arXiv:2611.09579v1stat.CO

Locally Adaptive Conformal Prediction for Polynomial Chaos Expansions

This work develops a locally adaptive jackknife+ conformal prediction framework for polynomial chaos expansions. Therein, the non-conformity scores are normalized with an auxiliary polynomial chaos expansion trained on the log-transformed leave-one-out residuals of the base model. We show that the coverage guarantee of jackknife+ conformal prediction carries over to the adaptive intervals if the leave-one-out auxiliary models are trained on nested leave-one-out residuals. For fixed-basis polynomial chaos expansions fitted by ordinary least squares, all required quantities are derived in closed form and the guarantee-preserving adaptive intervals are obtained from a single model fit, without any model retraining. For sparse polynomial chaos expansions, where multiple model fits are necessary, we propose a heuristic simplification that limits the computational overhead of adaptivity at the cost of the formal coverage guarantee. Numerical experiments on an academic benchmark with known heteroscedastic noise and on an engineering simulation dataset show that the suggested method attains the target coverage while providing intervals that adapt to local prediction uncertainty, therefore being considerably more statistically efficient. These advantages are particularly prominent in high-uncertainty regions of the input space, where the performance of non-adaptive methods degrades substantially.

arXiv:2611.08784v1stat.ML

Blind Noise Equalization for Large Data Matrices under General Variance Profiles

Low-rank approximation is a basic tool for inference from large and noisy data matrices. However, when noise variances differ across entries, a setting known as heteroskedasticity, the noise can distort the observed spectrum, producing spurious components and obscuring weak signals. To address these challenges, we develop BRINE (Blind Resolvent-based Iterative Noise Equalizer), a data-driven procedure that iteratively rescales rows and columns to equalize the average noise variance across both dimensions. Under suitable conditions, this procedure restores the Marchenko-Pastur law for the noise spectrum and can amplify signal components without prior knowledge of the signal or noise structure. A key idea underlying our approach is to equalize the diagonal entries of the data resolvent as a proxy for equalizing the unknown noise variances. At the population level, we identify a potential function that characterizes this equalization and motivates an iterative minimization procedure. We prove convergence of the iterations to the true equalizer for arbitrary positive variance profiles, with a geometric rate under mild conditions. The updates can be estimated directly from noisy data, and we establish probabilistic error bounds for the scaling factors. Simulations and experiments across diverse biological data modalities and preprocessing pipelines illustrate the effectiveness of our approach.

arXiv:2611.08541v1stat.ML

ELPAC: Endpoint-Anchored Latent Progression with Stage-Varying Multimodal Coordination

Aging cohorts increasingly combine imaging, plasma, and cognitive measurements, but most subjects provide only a partially observed cross-sectional snapshot, leaving latent stage, progression subtype, and stage-varying multimodal coordination to be inferred jointly. Existing methods estimate the cohort-level aging process, model covariance under observed covariates, or fuse modalities for prediction. However, they rarely target the joint problem of endpoint-anchored staging and progression-resolved residual coordination across modalities, even though subject staging, subtype assignment, and multimodal interplay are mutually dependent. We introduce ELPAC, a Bayesian framework with two interlocking constructions. The first integrates subtype trajectories backward from clinically reliable endpoint groups along biologically structured velocity components encoding modality-specific monotone direction and optional sequential ordering priors. The second factorizes residual multimodal variation around the inferred stage axis into sparse coordination axes whose loadings and activations expose which multimodal patterns are operative in each subtype and when they emerge, with stage-resolved variation supplying the auxiliary structure for conditional identifiability. We use a scalable amortized variational inference scheme that enables joint posterior inference at cohort scale with single-pass deployment on new subjects. On synthetic data and a harmonized ADNI+NACC/SCAN Alzheimer's cohort, ELPAC demonstrates accurate latent-structure recovery, well-calibrated prediction intervals, and biologically interpretable multimodal coordination programs that differ by subtype.

arXiv:2609.22183v2cs.LG

CleanScore: 음성 대조군과 민감도 경계를 활용한 블랙박스 벤치마크 감사

CleanScore: Black-Box Benchmark Audits with Negative Controls and Sensitivity Bounds

Public benchmark scores reflect skill, prior exposure to the questions, or both, and for most models the training data are unknown. CleanScore is a black-box audit: the estimator needs only a correct-or-incorrect score for each form, not training data or weights. Each benchmark question becomes a parent item with one public form and two independently written fresh forms that keep its numbers, facts and answer. The audit reports an interval for the public-form advantage, not a verdict, and a private control bank with an explicit transport radius provides a proxy for the ordinary form gap that a rewrite alone can produce. A registered controlled-exposure experiment, a preliminary validation, detects planted exposure in one of two model families. A registered audit of five open models on 200 GSM8K and 200 ARC-Challenge items (ARC with stem-only rewrites, scored by option log probabilities) finds no exposure-consistent advantage, and two reruns declared before they were run (GSM8K at a longer token budget, ARC scored from generated letters) leave that conclusion unchanged. For four of the ten model-benchmark pairs the upper end of the interval is about five points or less, and for none is it above 21 points. The registered claim rule is much less sensitive than the primary interval: it flags a planted ten-point advantage in 5% to 11% of audits. Registered positive controls show what a null can mean. In four models, each trained under four seeds, items leaked into fine-tuning gained 25 points on average over unleaked items on the leaked wording and 19 points on paraphrases the model never saw, so a substantial share of the effect, estimated at 76% (95% interval 50% to 113%), is invisible to a paraphrase audit. A null on surface form says much less about contamination than the name suggests.

arXiv:2608.21729v2stat.ML

밀도 비율 추정을 통한 사전분포 변경 가이던스

Guidance for Prior Change via Density Ratio Estimation

시뮬레이션 기반 추론(SBI)은 우도를 직접 계산하기 어려운(intractable) 시뮬레이터가 관여하는 과학 분야에서 매개변수 추론을 위한 핵심 프레임워크이지만, 상각(amortized) 생성 모델이 빠른 사후 추정을 제공하는 한편 학습 중 사용된 특정 사전분포에 종종 제한되어 사전 지식이 갱신될수록 유연성이 떨어진다. 이러한 사전분포 의존성을 완화하기 위해 PriorGuide가 추론 시점 가이던스 방법으로 도입되었으나, 다루기 어려운 formulation 때문에 역 전이 커널의 가우시안 근사와 사전분포 비율에 대한 가우시안 혼합 모델 적합에 의존하며, 둘 다 체계적 바이어스를 유발한다. 이러한 한계에 동기를 받아, 우리는 밀도 비율 추정(DRE)을 활용해 스코어 가이던스 항을 학습하고 추론 과정을 학습 시 사전분포로부터 효과적으로 분리하는 편향 없는(unbiased) 테스트 시점 가이던스 프레임워크를 제안한다. 또한 우리 프레임워크는 특정 밀도 비율 추정기에 무관(agnostic)하여 사전분포 변경을 다루는 일반적이고 유연한 프레임워크이다. 여러 과제에 걸친 실험 결과는 우리 방법이 대부분의 과제에서 C2ST와 MMD 기준으로 PriorGuide와 동등하거나 더 나은 성능을 보이며, 학습 사전분포와 목표 사전분포 간 겹침(overlap)이 제한적인 경우에도 견고함을 유지함을 보여준다. 더 나아가 행성 광곡선(planetary light-curve) 데이터로부터 매개변수 추론을 위한 베이즈 갱신(Bayesian updating)에 우리 방법을 적용했을 때도 강한 효과와 견고함을 보였다. 코드는 https://github.com/a-chenchen/dre-based-prior-guidance 에서 이용할 수 있다.

arXiv:2607.06089v2math.ST

타원형 마할라노비스 판별 분석을 위한 폐형식 분수 방사형 링크

Closed-form fractional radial links for elliptical Mahalanobis discriminant analysis

We study binary classification when the two classes are elliptical with a shared radial generator $g$. The log-likelihood ratio is then an additive function of the two squared Mahalanobis radii, with radial link $\varphi=\log g$. For Gaussian classes $\varphi$ is affine and the rule reduces to quadratic discriminant analysis (QDA); otherwise the Bayes link is not the identity. Instead of fitting this link with a spline smoother, we derive its family from the within-class law of the radius and estimate a finite fractional-power projection. We prove that the link is identifiable from the radius law, that the plug-in estimator is $\sqrt{n}$-consistent and asymptotically normal under finite-moment conditions, and that the induced classifier is asymptotically Bayes-optimal in an iterated sieve limit. For $d\ge2$ and non-nested class parameters, the identity link suffices only for affine (Gaussian) generators. The bridge, GAM membership and the affine direction of this dichotomy are verified in Lean 4. We compare against the global Mahalanobis-GAM of Ghosh et al. (2025), re-implemented with mgcv REML splines at equal input budget. On two UCI benchmarks the derived link ties with the fitted GAM. On six financial series from three asset classes, with validation that respects temporal dependence, it is never significantly worse than the fitted GAM: better on CAD/USD and tied on the other five. It beats QDA on the three heaviest-tailed series (oil $[+0.023,+0.069]$, S&P 500 $[+0.039,+0.125]$, JPY/USD $[+0.009,+0.047]$) and ties elsewhere. A rate simulation with a closed-form ground truth reproduces the $\sqrt{n}$ rate and, under covariance heterogeneity, QDA's risk floor against the derived link's vanishing excess risk. The derived link thus matches a tuned GAM without smoothing-parameter selection and improves on QDA where the generator departs from the Gaussian.

arXiv:2605.17778v2math.ST

Self-Distillation is Optimal Among Spectral Shrinkage Estimators

Self-distillation has emerged as a promising technique for improving model performance in modern machine learning systems. We develop the statistical foundations of self-distillation in high-dimensional linear regression by introducing and analyzing a broad class of estimators, namely spectral shrinkage estimators. We consider both spiked and general covariance matrices. For the former, we establish that when the covariance has $s$ distinct spikes, $s$-step self-distillation achieves optimal prediction among spectral shrinkage estimators, outperforming well-known estimators in statistics and machine learning. Moreover, we show that $s$ steps are necessary for optimality: any $(s-k)$-step distilled estimator is strictly suboptimal for $1 \leq k \leq s$. For general covariance matrices, we show that self-distillation approaches the optimal limiting prediction risk among continuous spectral shrinkage estimators as the number of distillation steps increases. We further develop a generalized cross-validation procedure for selecting the distillation parameters from data and establish an asymptotic oracle guarantee for the resulting estimator. We also study a federated approach where multiple data centers share spectral shrinkage estimators and a common server seeks to aggregate them to achieve optimal performance. In this case, we find that the best local rule again takes the form of self-distillation, though it differs from the optimal rule when data are hosted centrally on a single server. Together, our results explain why self-distillation improves predictive performance and provide a broader statistical framework that connects it to classical shrinkage-based methods.

arXiv:2605.05024v2stat.ML

Hypergraph Generation via Structured Stochastic Diffusion

Hypergraphs model higher-order interactions, but realistic hypergraph generation remains difficult because incidence, hyperedge-size heterogeneity, and overlap structure are not faithfully captured by pairwise reductions. We propose \HEDGE, a generative model defined directly on relaxed incidence matrices via a structured stochastic diffusion. The forward process combines a hypergraph-specific two-sided heat operator with an Ornstein--Uhlenbeck component, preserving structure-aware noising near the data while yielding an explicit Gaussian terminal law. Conditional on an observed hypergraph, this forward process is linear-Gaussian, so conditional means, covariances, scores, and reverse-drift targets are available in closed form. We therefore learn a permutation-equivariant state-only reverse-drift field in incidence space by regressing onto exact conditional targets, and generate samples by simulating a learned reverse-time SDE from the Gaussian base law. We establish exactness in the ideal state-only setting together with finite-horizon stability guarantees, and empirically show improved hypergraph generation quality relative to strong baselines.

arXiv:2602.07915v3cs.LG

CausalCompass: Evaluating the Robustness of Time-Series Causal Discovery in Misspecified Scenarios

Causal discovery from time series is a fundamental task in machine learning. However, its widespread adoption is hindered by a reliance on untestable causal assumptions and by the lack of robustness-oriented evaluation in existing benchmarks. To address these challenges, we propose CausalCompass, a flexible and extensible benchmark framework designed to assess the robustness of time-series causal discovery (TSCD) methods under violations of modeling assumptions. To demonstrate the practical utility of CausalCompass, we conduct extensive benchmarking of representative TSCD algorithms across eight assumption-violation scenarios. Our experimental results indicate that no single method consistently attains optimal performance across all settings. Nevertheless, in most scenarios, the methods exhibiting superior overall performance are deep learning-based approaches. We further provide hyperparameter sensitivity analyses to deepen the understanding of these findings. We additionally conduct ablation experiments to explain the strong performance of deep learning-based methods under assumption violations. We also find that NTS-NOTEARS relies on standardized preprocessing in practice, performing poorly in the vanilla setting but exhibiting strong performance after standardization. Finally, our work aims to provide a comprehensive and systematic evaluation of TSCD methods under assumption violations, thereby facilitating their broader adoption in real-world applications. The user-friendly implementation, documentation and datasets are available at https://github.com/huiyang-yi/CausalCompass.

arXiv:2509.17228v2cs.LG

Causal Representation Learning from Multimodal Clinical Records under Non-Random Modality Missingness

Clinical notes contain rich patient information, such as diagnoses or medications, making them valuable for patient representation learning. Recent advances in large language models have further improved the ability to extract meaningful representations from clinical texts. However, clinical notes are often missing. For example, in our MIMIC-IV cohort, 35.15% of patients have no eligible discharge summaries. In such cases, representations can be learned from other modalities such as structured data, chest X-rays, or radiology reports. Yet the availability of these modalities is influenced by clinical decision-making and varies across patients, potentially producing modality missing-not-at-random (MMNAR) patterns. We propose a causal representation learning framework that leverages observed data and informative missingness in multimodal clinical records. It consists of: (1) an MMNAR-aware modality fusion component that integrates structured data, imaging, and text while conditioning on missingness patterns to capture patient health and clinician-driven assignment; (2) a modality reconstruction component with latent alignment to preserve modality information in the learned representation; and (3) a multitask outcome prediction model with a rectifier that corrects for residual bias from specific modality observation patterns. Comprehensive evaluations across MIMIC-IV and eICU show consistent gains over the strongest baselines, achieving relative AUC improvements of up to 4.3% for hospital readmission and 2.7% for ICU admission.

arXiv:2310.07801v2cs.CV

Stay on the Manifold: Hierarchical Bayesian Trajectory-Family Transfer for Sparse-Target Reconstruction and Sampling

Where should reconstructions and new samples lie when a few observations reveal only fragments of a trajectory? We propose hierarchical Bayesian trajectory principal-manifold sampling (HB-TPM), which learns a family of smooth curves from ordered source tasks and adapts it to sparse targets. For unordered vectors, alternating progression assignments and coefficient updates estimate trajectory structure between and beyond the observed shots under the learned source family. Observation-aware extensions connect the trajectory prior to photon arrivals and image-encoded cardiac cycles. Our analysis characterizes the coefficient directions supplied by the prior and bounds sampling error conditional on curve recovery. Across 30 held-out vector tasks, HB-TPM reduces mean sliced-Wasserstein distance by 21.0-41.8% relative to a source-adapted VAE. Prior-strength and sampling ablations identify coefficient shrinkage as the main supported mechanism. On 100 simulated photon records, adaptation lowers mean relative trajectory RMSE by 50.2% versus target-only fitting and 23.6% versus source-mean prediction. Across three training seeds and five folds on 98 cardiac sequences, HB-TPM with latent-MSE training achieves the best mean reconstruction among the tested configurations. Together, the results demonstrate trajectory-family transfer for sparse-target reconstruction and trajectory-centered sampling when source and target tasks share comparable progression structure.