PyTorchKR Papers

최신 논문

총 171편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2611.10388v1stat.ME

Causal Discovery Grounding and the Naturalness Assumption

Standard causal discovery postulates that structure can be described by a directed acyclic graph (DAG), which can be partially reconstructed by using the faithfulness principle. According to this principle, conditional independencies in the observational distribution correspond exactly to those encoded in the unknown DAG. However, such algorithms are challenged by the fact that, without even stronger assumptions, approximate failures of faithfulness are empirically undistinguishable from exact ones. This leaves an inferential gap which can lead to major implications to causal effect identification. We introduce a different perspective, not reliant on exact independencies. By considering the specialized problem of discovering whether unmeasured confounding can be ruled out for a particular cause-effect pair, we introduce a principle we call naturalness, in which the strength of observational associations bounds the strength of causal dependencies. The bounding function is unknown, with degrees of freedom not present in faithfulness. We model uncertainty about this function using a Bayesian framing which allows for the use of experiments from separate studies, grounding an adaptive causal discovery principle on historical experimental data. We demonstrate its usefulness in partial identification of causal effects, and discuss how the promising results can motivate new principles for general causal discovery.

arXiv:2611.10373v1stat.ME

SAMBoost: Spatially Aware Mesh Boosting

We propose SAMBoost (Spatially Aware Mesh Boosting), a gradient boosting method that combines tree-based covariate modelling with a stochastic partial differential equation (SPDE) latent spatial field through a single penalised objective. At each boosting iteration, the predictor is decomposed into separate covariate and spatial components. The method is motivated by applications in which the spatial effect is not a nuisance to be removed but a quantity of interest that must be recovered alongside the covariates. In insurance, for example, practitioners need both flexible nonlinear covariate effects and an interpretable geographic risk surface that can be reported on its own. Existing approaches often involve trade-offs, offering flexible covariate modelling but no explicit spatial field, rigorous spatial inference with a fixed additive covariate structure, or a combination of both at computational costs that limit practicality for large datasets. SAMBoost addresses these goals within the boosting loop, and the underlying mesh produces a sparse precision structure whose cost is largely independent of sample size. On simulations and on an open benchmark, SAMBoost matches or improves on the predictive accuracy of joint Gaussian process and tree boosting approaches. It is also faster and recovers the spatial field and covariate effects more accurately. When covariate effects include interactions, it outperforms additive Bayesian spatial models. Applied to a flood insurance portfolio with 4.9 million records, SAMBoost trains on the full training set without subsampling and runs about sixteen times faster than a Bayesian reference, with a consistent spatial risk surface. A public Python implementation is available.

arXiv:2610.39070v2cs.AI

When the Same Prompt Gives Different Answers (and the Same Answer Isn't Always Right): A Taxonomy of Failure Mechanisms, Testing, and Mitigation Strategies for LLM-Assisted Evidence Synthesis Read-and-Evaluate Tasks

LLM read-and-evaluate tasks include study screening, data extraction, risk-of-bias assessment, and critical appraisal in the evidence synthesis field. These task differ in the amount of LLM interpretation they require. Some tasks involve retrieving relatively simple information, such as an author name, sample size, or publication year. Others require the LLM to read source material, interpret it against predefined criteria, and make a judgement. Researchers should assess whether LLM judgements are accurate, supported by appropriate source evidence, and reproducible under specified conditions. A correct judgement does not necessarily indicate that the underlying evaluation process is reliable. We developed a taxonomy for identifying mechanisms that may compromise these properties and for informing the selection and reporting of appropriate evaluation methods. The taxonomy outlines potential ways that LLM judgments from read-and-evaluate tasks can fail, called TRAILS (Taxonomy of Reliability And Instability in LLM-assisted evidence Synthesis). The TRAILS taxonomy addresses two complementary questions. First, what mechanisms can cause the same read-and-evaluate task to produce different judgements across repeated LLM runs? Second, what mechanisms can systematically affect the judgement or required output in a way that is not apparent from the judgement itself? This study presents 38 mechanisms and identifies the deployment settings in which each can operate, approaches for testing them, and mitigation strategies. We also show why repeatability or accuracy does not rule out a hidden judgement failure and why human evaluation should assess whether a judgement is grounded in the supplied evidence, independent of inappropriate external influence, and produced through a process that can be examined and verified.

arXiv:2610.36617v2cs.LG

Balanced State Subspaces with Spectral Rank Allocation for Mamba-2 State Compression

Mamba-2 compresses all context into a fixed-size recurrent state that dominates decoding memory and bandwidth. Existing compressors quantize state values or prune coordinates with per-channel scores. Trained states are nearly rank-one and bias-like, aligned with the mean write; but the output's energy spans an order of magnitude more directions. Per-channel scores see only the diagonal of the write--read operator, so they cannot select the subspace that governs the input--output map, which is a rotation of the coordinates. We therefore propose \method{}, which keeps that subspace by applying balanced truncation exactly via Mamba-2's scalar-identity transition. Layers differ just as sharply: most are bias-dominated, while the few that carry long-range associations are the flattest and need most of their state. We therefore introduce \spar{}, which redistributes a fixed total rank budget across layers according to the Hankel spectrum. On Mamba-2 checkpoints from 130M to 2.7B, \method{}+\spar{} halves the state at an average cost of $0.3$ WikiText-2 perplexity points, against $1.6$ for GHOST, while retaining $97\%$ of the dense model's five-digit in-context recall. At a quarter of the state, giving every layer the same rank keeps $5\%$ of that recall and \spar{} $77\%$, for a $7\%$ perplexity cost.

arXiv:2607.26346v2math.ST

Toward a Unified Statistical Theory of Multi-View Pretraining for Neural Knowledge Graph Learning

Knowledge graph learning offers a powerful framework for representing and inferring structured knowledge, with broad practical applications. Two obstacles limit current methods: labeled triples are scarce for each entity and relation, which makes expressive models hard to train, and scoring functions are designed ad hoc, which limits their generalizability and leaves them without theoretical grounding. We address both obstacles with a theoretically grounded framework, with end-to-end guarantees, that extends and subsumes existing methods. The framework proceeds in two stages: multi-view unsupervised pretraining that integrates heterogeneous sources of entity side information into a shared representation, followed by supervised learning across multiple relation types. We establish a unified nonasymptotic risk bound that combines the subspace recovery error of pretraining and the sample complexity of the labeled data in a single guarantee, quantifying formally how large-scale unlabeled data from multiple views benefits downstream knowledge prediction. Synthetic experiments validate each component of the theory, and experiments on large-scale real-world knowledge graph benchmarks confirm the effectiveness of our approach.

arXiv:2607.09645v2stat.ML

Deep Gaussian Processes on Directed Acyclic Graphs

Many real-world processes can be represented as compositions of functions along a directed acyclic graph (DAG). In causal modelling, these correspond to the underlying mechanisms; in engineering, to multiple fidelity levels; and in gene-regulatory networks, to transcription factors. These functions are partially observed across the DAG, with noisy and heterogeneously sampled measurements, posing significant challenges for reconstruction, uncertainty propagation, and inference. To tackle these challenges, we place priors over functions and naturally arrive at Deep Gaussian Processes over DAGs. We theoretically study their prior-collapse behaviour and the effect of graph topology and intermediate observations on the preservation of information. We obtain almost-sure lower bounds on the asymptotic frequency of depths at which the distinction between inputs is preserved, identify broad kernel classes for which these hold, and prove an observation by \citep{dunlop2018} on the role of input connections. We offer a structured variational approximation that retains graph dependencies, preserves compositional uncertainty, and captures the explaining-away behaviour of colliders. Finally, we empirically validate our theory and methodology at scale and under DAG misspecification. We then model a latent-collider DAG, a protein signalling network, and a multi-fidelity \mbox{heavy-ion} collision emulation task, attaining state-of-the-art performance while recovering low-fidelity contributions and yielding interpretability over the simulator hierarchy.

arXiv:2606.00436v2stat.ME

Weighted Conformal Clustering

Clustering is a central tool for discovering latent structure in unlabeled data; yet modern clustering pipelines often end with a hard assignment of each observation to a cluster without rigorous measures of assignment uncertainty. We propose a novel weighted conformal approach for constructing valid confidence sets for cluster labels. The key difficulty is that the labels available for calibration are not observed ground-truth labels, but synthetic labels produced by a data-dependent clustering algorithm. Our method develops a conformal inference algorithm that corrects the resulting mismatch with the latent target labels through weights by formulating conformal clustering as a conditional label-distribution shift problem. We first derive an oracle procedure that attains finite-sample marginal coverage and then develop a computationally tractable and implementable version using estimated conditional label probabilities and novel augmented calibration. We show that the coverage of the estimated-weight procedure depends on the estimator, giving an explicit bound on the loss relative to the nominal level. Empirical studies demonstrate that the proposed weighted approach offers improvements over the recently proposed split conformal clustering procedure in terms of informative confidence set size, especially in nonlinear and high-dimensional clustering applications.

arXiv:2605.02205v3stat.ME

2D Stability Selection: Design Jittering to Exclude Correlated Irrelevant Variables

Stability Selection finds the variables that are chosen consistently across random subsamples of the data, but it can consistently choose the wrong ones. An irrelevant variable that is strongly correlated with relevant variables can be selected in many subsamples, because resampling the observations does not change the variables themselves. We call such a variable a confounder; the term refers only to this correlation and not to causal confounding. We propose 2D Stability Selection, which also adds random noise to the design matrix, at increasing noise levels, and keeps the variables that are selected throughout. On average, the added noise acts as a ridge penalty, so each fit is a noisy elastic net. This changes the conditions under which the Lasso selects the correct variables: it can restore them when a confounder would otherwise be chosen, and it can also make selection worse. For uncorrelated relevant variables we give an exact criterion for the noise levels at which the population elastic-net irrepresentable condition holds, recovery guarantees for a single fit and for the aggregated procedure, and, for thresholds above $1/2$, a bound on the expected number of false positives (under i.i.d.\ observations; its closed form also needs exchangeable irrelevant variables). In simulations, the method excludes the confounders that Stability Selection includes. In the simulated design, an elastic net used without any noise, with a fixed ridge weight or along a deterministic path, excludes the confounders as well, which suggests that for the Lasso the ridge effect, not the randomness of the noise, is responsible.

arXiv:2604.22391v3stat.ML

Conformalized Super Learner

The Super Learner (SL) is a widely used ensemble method that combines point predictions from a library of learners based on their predictive performance. Interval predictions are of considerable practical interest because they allow uncertainty in predictions produced by an individual learner or an ensemble to be quantified. Several methods have been proposed for constructing interval predictions based on the SL, however, these approaches are typically justified using asymptotic arguments or rely on computationally intensive procedures such as the bootstrap. Conformal prediction (CP) is a machine learning framework for constructing prediction intervals with finite-sample and asymptotic coverage guarantees under mild conditions. We propose coupling CP with the SL through a natural construction that mirrors the original SL framework, using individual learner weights and combining learner-specific conformal prediction sets via a weighted majority vote. We characterize the properties of the resulting SL-based prediction intervals for continuous outcomes. We cover settings under exchangeability, and data-generating mechanisms exhibiting heteroscedasticity, sparsity, and other forms of distributional heterogeneity. A comprehensive simulation study shows that the conformalized SL achieves valid finite-sample coverage with competitive performance relative to the true data-generating mechanism. We present an application of this methodology to predict creatinine levels using socio-demographic, biometric, and laboratory measurements. This example demonstrates the benefits of an ensemble with carefully selected learners designed to capture key aspects of complex regression functions, including non-linear effects, interactions, sparsity, heteroscedasticity, and robustness to outliers.

arXiv:2604.20568v3cs.LG

Amortized Vine Copulas for High-Dimensional Density and Information Estimation

Modeling high-dimensional dependencies while keeping likelihoods tractable re- mains challenging. Classical vine copulas provide an explicit dependence structure but require repeated, costly bivariate fitting. We propose Vine Denoising Copula (VDC)1, an amortized estimator for simplified vines on continuous data. VDC trains a single bivariate denoising network on simulated copulas and reuses it across all vine edges. For each edge, the network maps an empirical histogram to a density grid, and iterative proportional fitting approximately enforces uniform grid marginals. The predicted densities and cached conditional transforms support stan- dard vine recursion, likelihood evaluation, sampling, and information estimation. This retains the classical vine factorization while replacing repeated per-edge fitting with reusable neural inference. Across synthetic and real-data benchmarks, VDC combines accurate bivariate density estimation, competitive information estimates, and faster high-dimensional vine fitting after pretraining. Performance varies across datasets; finite grid resolution and conditional prediction remain limitations. The explicit fitted structure supports edge- and tree-level information decomposition, making repeated dependence analysis more practical.

arXiv:2603.14094v3stat.ML

Maximin Robust Bayesian Experimental Design

We address the brittleness of Bayesian experimental design under model misspecification by formulating the problem as a max--min game between the experimenter and an adversarial nature subject to information-theoretic constraints. We demonstrate that this approach yields a robust objective governed by Sibson's $\alpha$-mutual information~(MI), which identifies the $\alpha$-tilted posterior as the robust belief update and establishes the R\'enyi divergence as the appropriate measure of conditional information gain. To mitigate the bias and variance of nested Monte Carlo estimators needed to estimate Sibson's $\alpha$-MI, we adopt a PAC-Bayes framework to search over stochastic design policies, yielding rigorous high-probability lower bounds on the robust expected information gain that explicitly control finite-sample error.

arXiv:2602.18958v3stat.ME

Structure-Adaptive Estimation of Heterogeneous Treatment Effects with Kernel Methods

We study conditional average treatment effect (CATE) estimation from observational data in regimes with potentially weak overlap, when the nuisance response functions lie in a reproducing kernel Hilbert space (RKHS), but the contrast function is structurally simpler. We consider three forms of such simplicity: (i) membership in a lower-complexity RKHS with faster eigenvalue decay, (ii) a source condition relative to the nuisance kernel, or (iii) dependence on a known low-dimensional covariate representation. We develop a unified two-stage kernel ridge regression procedure that remains valid under weak overlap and attains minimax rates determined by the complexity of the contrast function. The procedure avoids propensity-score estimation and separate nuisance convergence-rate assumptions. We also show that a simple model-selection step over candidate contrast spaces and regularization levels satisfies an oracle inequality, enabling adaptation to unknown CATE regularity.

arXiv:2602.04042v3cs.LG

Partition Tree: Conditional Density Estimation over General Outcome Spaces

We propose Partition Tree, a tree-based framework for conditional density estimation over general outcome spaces that supports both continuous and categorical variables within a unified formulation. Our approach models conditional distributions as piecewise-constant densities on data-adaptive partitions and learns trees by greedily minimizing a surrogate for conditional log-loss. This yields a scalable alternative to existing probabilistic trees that does not make parametric assumptions about the target distribution. We further introduce Partition Forest, an ensemble extension obtained by averaging conditional densities. Empirically, we demonstrate improved probabilistic prediction over CART-style trees and competitive performance compared to state-of-the-art probabilistic tree methods and Random Forests.

arXiv:2601.02322v3stat.ME

Environment-Adaptive Covariate Selection: Learning When to Use Spurious Correlations for Out-of-Distribution Prediction

A common approach to out-of-distribution prediction restricts models to causal or invariant covariates to avoid spurious associations that may change across environments. Despite its theoretical appeal, this strategy can underperform empirical risk minimization when only a subset of the causal parents of the outcome is observed. In such settings, non-causal covariates can serve as proxies for unobserved causal parents and improve prediction when the proxy relationship is stable, but they can hurt when shifts disrupt that relationship. Thus, the optimal covariate set can depend on the specific shift encountered. Because different shifts can leave distinct signatures in the covariate distribution, we propose an environment-adaptive covariate selection algorithm that chooses which covariates to use in each environment based on summaries of its unlabeled covariates. The summaries may be hand-crafted or learned from multi-environment data, and prior causal knowledge can be incorporated as constraints. We give sufficient conditions under which such summaries are informative in linear causal models, and explain when adapting only the covariate set is preferable to letting every coefficient vary with the environment. In simulations and two applications, adapting the covariate set improves prediction over static baselines, including causal and invariant covariate selection.

arXiv:2510.00569v2stat.ML

Guaranteed Noisy CP Tensor Recovery via Riemannian Optimization on the Segre Manifold

Recovering a low-CP-rank tensor from noisy linear measurements is a central challenge in high-dimensional data analysis, with applications spanning tensor PCA, scalar-on-tensor regression, tensor-on-tensor regression, and beyond. We exploit the intrinsic geometry of rank-one tensors by casting the recovery task as an optimization problem over the Segre manifold, the smooth Riemannian manifold of rank-one tensors. This geometric viewpoint yields two powerful algorithms: Riemannian Gradient Descent (RGD) and Riemannian Gauss-Newton (RGN), each of which preserves feasibility at every iteration. Under mild noise assumptions, we prove that RGD converges at a local linear rate, while RGN exhibits an initial local quadratic phase followed by linear refinement to the statistical error. Extensive synthetic experiments validate these convergence guarantees and demonstrate the practical effectiveness of our methods.

arXiv:2508.00110v3stat.ML

funOCLUST: Clustering Functional Data with Outliers

Functional data present unique challenges for clustering due to their infinite-dimensional nature and potential sensitivity to outliers. An extension of the OCLUST algorithm to the functional setting is proposed to address these issues. The approach leverages the OCLUST framework, creating a robust method to cluster curves and trim outliers. The methodology is evaluated on both simulated and real-world functional datasets, demonstrating strong performance in clustering and outlier identification.

arXiv:2505.20536v2stat.ML

Covariate-Adjusted Deep Causal Learning for Heterogeneous Panel Data Models

This paper studies the task of estimating heterogeneous treatment effects in causal panel data models with covariate effects. We propose a novel Covariate-Adjusted Deep Causal Learning (CoDEAL) framework that cohesively deals with the underlying heterogeneity and nonlinearity of both panel units and covariate effects. CoDEAL integrates nonlinear covariate effect components (parameterized by a feed-forward neural network) with nonlinear factor structures (modeled by a multi-output autoencoder) to form a heterogeneous causal panel model. The nonlinear covariate component flexibly captures complex covariate influences on outcomes, and the nonlinear factor decomposition enables CoDEAL to effectively capture both cross-sectional and temporal dependencies inherent in the data panel. This latent structural information is subsequently integrated into a customized matrix completion algorithm, thereby facilitating more accurate counterfactual imputation. Moreover, the use of a multi-output autoencoder explicitly accounts for heterogeneity across units and enhances the model interpretability of the latent factors. We establish theoretical guarantees on the convergence of the estimated counterfactuals and demonstrate the compelling performance of the proposed method using extensive simulation studies and real-world data applications.

arXiv:2605.16757v3cs.AI

NeuroMAS: Multi-Agent Systems as Neural Networks with Joint Reinforcement Learning

Multi-agent language systems are often built as hand-designed workflows, where agents are assigned semantic roles and communication protocols are specified in advance. We propose NeuroMAS, a framework that treats a multi-agent language system as a trainable and scalable neural-network-like architecture with hidden textual intermediate signals. In NeuroMAS, language-agent nodes are role-free but structure-aware: the topology determines how information can flow, while reward-based training determines how nodes communicate, specialize, and coordinate. This formulation shifts multi-agent design from workflow engineering toward architecture design, where depth, width, connectivity, and growth protocol become scalable sources of capability. We provide a theoretical perspective showing why such modular text-valued computation can be parameter-efficient when tasks admit stable decompositions. Experiments show that NeuroMAS improves over both inference-time collaboration methods and trained baselines, and that its gains are not explained by trainable parameter count alone. We further find that organizational scaling is path-dependent: larger systems can fail when trained from scratch, but become stronger when grown progressively from smaller trained systems. These results suggest that learned multi-agent organization is a promising scaling axis for language-model reasoning and generation.

arXiv:2611.09698v1stat.CO

A Generalized Bayesian Approach to Clustering with Dimension Reduction

Generalized Bayes product partition models provide a probabilistic counterpart to loss-based clustering. They update a prior over partitions through an exponentiated clustering loss rather than a likelihood, so allocation uncertainty can be quantified without a full generative model for the data. However, their performance depends on the space in which the loss is evaluated. For high-dimensional observations, a squared Euclidean loss on the raw features can be dominated by irrelevant coordinates or fail to reflect cluster structure lying near a nonlinear low-dimensional manifold. We propose an extension in which an autoencoder and the partition are learned together, so that the space in which the clustering loss is evaluated is shaped by the clustering itself. Our generalized posterior combines a reconstruction loss, which keeps the low-dimensional representation faithful to the observed data, with a within-cluster sum of squares in the latent representation, which encourages cluster separation. Posterior computation alternates between Gibbs sampling over partitions and gradient-based updates of the autoencoder network, with the learning rate estimated jointly. We illustrate the approach through simulation studies and applications to four benchmark datasets.

arXiv:2611.09579v1stat.CO

Locally Adaptive Conformal Prediction for Polynomial Chaos Expansions

This work develops a locally adaptive jackknife+ conformal prediction framework for polynomial chaos expansions. Therein, the non-conformity scores are normalized with an auxiliary polynomial chaos expansion trained on the log-transformed leave-one-out residuals of the base model. We show that the coverage guarantee of jackknife+ conformal prediction carries over to the adaptive intervals if the leave-one-out auxiliary models are trained on nested leave-one-out residuals. For fixed-basis polynomial chaos expansions fitted by ordinary least squares, all required quantities are derived in closed form and the guarantee-preserving adaptive intervals are obtained from a single model fit, without any model retraining. For sparse polynomial chaos expansions, where multiple model fits are necessary, we propose a heuristic simplification that limits the computational overhead of adaptivity at the cost of the formal coverage guarantee. Numerical experiments on an academic benchmark with known heteroscedastic noise and on an engineering simulation dataset show that the suggested method attains the target coverage while providing intervals that adapt to local prediction uncertainty, therefore being considerably more statistically efficient. These advantages are particularly prominent in high-uncertainty regions of the input space, where the performance of non-adaptive methods degrades substantially.