Uncertainty quantification in prediction is a central problem in statistics and machine learning. Conformal prediction provides a flexible approach to constructing prediction sets with valid marginal coverage under exchangeability. In this paper, we investigate conformal prediction for both one-step and multi-step forecasting from a single stationary time series, addressing temporal dependence and the additional challenges of joint prediction across multiple horizons. We propose a kernel-based framework, called Time series Residual Analysis for Calibration with Kernels (TRACK), for estimating conditional quantiles of residual scores and constructing prediction sets from rolling-window forecasters. The framework accommodates heterogeneous uncertainty across horizons and accounts for dependence induced by rolling estimation, while allowing predictor memory and forecast horizons to grow with the sample size. Our finite-sample bounds quantify how temporal dependence and overlapping score windows affect calibration and coverage. We further derive bounds for conditional quantile estimation and pointwise conditional coverage discrepancy. The guarantees extend to more general predictors that admit an accurate rolling-window approximation. We also derive matching minimax bounds over classes of known score maps, clarifying how overlapping score windows affect calibration accuracy. Experimental results from simulations and applications to wind energy generation and bike sharing demand show that our methods improve regional conditional coverage while maintaining competitive interval widths.
Randomized trial findings are routinely generalized to broader target populations to inform decisions in medicine and public policy. When the trial sample is misaligned with the target population, generalized effect estimates can yield suboptimal decisions. We characterize the target subgroups that are underrepresented in the trial cohort in time-to-event settings. Here, underrepresentation arises in two distinct ways: baseline underrepresentation, when a subgroup's covariate profile is rare in the trial relative to the target, and time-varying underrepresentation, when differential censoring erodes a subgroup over follow-up. Existing diagnostics either conflate these mechanisms or apply only to the baseline case. We show that the asymptotic variance of the efficient generalizability estimator admits a decomposition in which only two terms can diverge as representation weakens: a sampling-weight term capturing baseline misalignment and a censoring-weight term capturing differential loss to follow-up. Underrepresented subgroups are therefore those with imprecise generalized estimates, and the larger of the two terms gives the responsible channel. We use this decomposition in TRACE (Temporal Rashomon Analysis of CEnsored data), which combines a self-normalized variance objective with a tree-based search to localize underrepresented subgroups, attribute each to baseline mismatch or censoring, and find the follow-up intervals where censoring drives the variance inflation. We solve the joint problem with a parametric dynamic program nested inside a Rashomon ensemble of near-optimal trees. In synthetic benchmarks, TRACE recovers the planted underrepresented regions. In a case study generalizing the National Job Training Partnership Act (JTPA) Study to U.S. JTPA-eligible adults, restricting to reliably generalizable subgroups narrows the 95% confidence interval for the refined estimand by 60%.
Representations learned from electronic health record (EHR) data encode patient physiology together with the clinical process that decides what is measured, when, and for whom. We formalize the data-generating process with latent physiology, ordering intent, laboratory values that exist only when measured, a learned representation, and clinical environments, and analyze two strategies for undoing this entanglement. First, no division of the information carried by laboratory "value" and "ordering" channels is identified from a single dataset: when ordering depends on the analyte, its distribution among untested patients is not recoverable, and a sentinel code for the undefined recorded value makes it carry all the entropy of the ordering indicator, fixing partial information decomposition (PID) for every redundancy measure satisfying the Williams-Beer axioms and invariant under injective re-encoding. Second, cross-site invariance of representation-conditional outcomes is a falsifiable implication of biological invariance taken jointly with exclusion and representation-sufficiency conditions, but not an identification strategy: we construct observationally equivalent models that disagree about biological invariance, whether the invariance test fails or passes. For binary physiology this happens exactly when case mix differs, the representation is lossy, and physiology affects the outcome. A worked example yields a 9.6-percentage-point cross-site gap under an unchanged biological mechanism. Tests performed for every patient remove the ordering gate, and measured tests can restore identification: biological invariance is identified when a test exactly summarizes outcome-relevant physiology, or, for binary physiology, when two tests with independent errors are available at sites with different case mix. Cross-site analyses should be reported as tests of a stated joint hypothesis.
We study the problem of learning directed acyclic graphs from observational data in nonlinear models and provide finite-sample, nonasymptotic statistical guarantees in these models. Although widely studied, statistical guarantees for gradient-based methods in nonlinear models are missing from the literature. Under a generalized linear model for each node, we analyze a global nonsmooth objective combining an exact acyclicity constraint with a quasi-minimax concave penalty, which can be optimized via standard gradient-based solvers. We establish finite-sample guarantees under this model, including the most widely used logistic and Poisson models. Experiments corroborate structure recovery accuracy and computational performance, including both standardized data and negative controls as baselines.
Unmeasured confounders are a key obstacle to accurately estimating treatment effects in observational studies, and in multilevel data, they often arise at the cluster level. Flexible machine learning methods cannot directly resolve such unmeasured confounding. To address this, we propose the variational autoencoder-augmented R-learner (VAE-R), which leverages variational autoencoders (VAEs) to infer unmeasured cluster-level covariates from the within-cluster dependence of outcome and treatment residuals, and uses the resulting adjustments to estimate the conditional average treatment effect (CATE) and the average treatment effect (ATE). The VAE decoder is specified through an offset likelihood so that the latent variables capture only the cluster-level variation left unexplained by measured covariates. VAE-R accommodates unmeasured cluster-level confounders in observational studies and unmeasured cluster-level predictors of the outcome in multisite randomized trials. We also provide data-driven strategies for selecting the two hyperparameters of the VAE, the latent dimension size and the Kullback-Leibler weight. Our simulation study shows that VAE-R nearly eliminates the bias of the ordinary R-learner's CATE and ATE estimates in multilevel observational data and improves their precision in multisite randomized trials. Finally, we illustrate VAE-R with two datasets, one from a multilevel observational study and the other from a multisite randomized trial.
Aligning two models without paired data requires a distance that both share, yet existing methods use Euclidean or spherical geometry. This is either an arbitrary choice or imposed by their loss function. We show that, under a precise form of the manifold hypothesis, the pullback of the data metric gives such a distance for any two models of one data-generating process. This holds whether the models are encoders, decoders or both, and whatever their architecture, loss, prior or training data. The alignment map preserves this distance and the distribution of representations, and is identifiable if and only if the data distribution breaks every symmetry of the data manifold. We show that this symmetry breaking holds generically. Identifiability makes the alignment coupling the unique optimum of Gromov--Wasserstein and of its weighted combinations with a linear program. The linear program alone has the same optimum if almost every data point has a distinct distribution of distances. The coupling, and with it the alignment map, is recovered exactly from permuted samples, even without identifiability, and consistently from independent ones. Our method, Neural Gromov--Monge Alignment (NGMA), uses no pairs or labels and outperforms every unpaired and label-informed baseline. It relies on pullback distances to align image models with different priors and single-cell models trained on disjoint cells, where Euclidean distances fail.
Tabular foundation models based on prior-data fitted networks (PFNs) achieve strong predictive performance, but their value for causal inference is unclear. We evaluated whether PFNs, which predict through in-context learning, could match or outperform conventional nuisance learners in supporting accurate causal estimation and reliable uncertainty quantification (inference). Learners modeled propensity scores and outcome regressions within doubly robust estimators of average causal effects, with and without cross-fitting, across synthetic and semi-synthetic settings with varying confounding complexity and effect heterogeneity. Under simple parametric data-generating processes, the PFNs were competitive with correctly specified parametric regressions and ensembles containing them. In more complex settings, they outperformed all conventional learners evaluated, including richly specified Super Learner ensembles, for both accuracy and uncertainty quantification. As sample size increased, only the two PFN models (TabPFN and Mitra) consistently approached nominal confidence interval coverage, whereas under-coverage persisted for conventional learners. In some settings, PFNs achieved near-nominal coverage and well-calibrated standard errors even without cross-fitting. At the smallest sample size (n=200) and in the higher-dimensional semi-synthetic settings, PFNs and other flexible learners exhibited poor coverage without cross-fitting. For PFNs, this appeared to be driven by propensity score overfitting consistent with in-sample memorization of treatment labels. These findings highlight the promise of prior-data fitted networks as in-context nuisance learners for causal estimation, while underscoring the need for further evaluation in challenging settings.
Modern observational biomedical datasets contain increasingly extensive baseline and outcome measurements, yet detecting a clinically meaningful treatment effect remains difficult because investigators must identify an outcome that captures treatment response while adjusting for confounding distributed across numerous noisy baseline variables. We introduce SCOUT (Score-based Causal Outcome Learning Using Treatment signals), a framework that learns a clinically interpretable, treatment-responsive outcome while adjusting for confounding throughout the learning process. SCOUT constructs low-dimensional adjustment representations of high-dimensional baseline measurements to estimate the average treatment effect (ATE) and conditional average treatment effect (CATE) for the learned outcome. Importantly, the final adjustment space is at most two-dimensional, allowing treatment-effect heterogeneity to be estimated in this space rather than in the full high-dimensional covariate space. SCOUT also introduces a fourth-order statistic for testing treatment-effect heterogeneity that emphasizes unusually large deviations from the ATE. In synthetic and semi-synthetic experiments, the algorithm achieved competitive ATE estimation accuracy and substantially lower CATE estimation error. In an observational study of patients with bipolar disorder, SCOUT identified a clinically recognizable lithium-associated adverse-effect profile, detecting both a significant ATE (Holm-adjusted $p$ = 0.0026) and significant treatment-effect heterogeneity (Holm-adjusted $p$ = 0.033).These results demonstrate how jointly learning outcomes and controlling for confounding can facilitate the discovery of treatment effects in high-dimensional observational data.
Recursive self-improvement offers a promising path toward AI agents that improve through experience by refining their own harnesses without retraining their underlying large language models (LLMs). Making this self-improvement process reliable, however, requires more than generating new updates: agents must decide which changes are worth keeping. A greedy gate that accepts updates based solely on higher validation scores can mistake an observed gain for genuine progress and lead to performance degeneration, whereas overly conservative statistical gates may reject beneficial updates and hinder performance improvement. We introduce Trustworthy Recursive Updates with Statistical Testing (TRUST), a framework that adapts an online multiple hypothesis testing procedure to control the rate of false evolution across rounds, with an LLM analyzing historical context to help determine how much evidence is required to update the incumbent. Under interpretable assumptions, we prove that TRUST limits the proportion of accepted updates that result in unacceptable performance loss. We further derive a lower bound on the final performance gain. Across the benchmarks HotpotQA, DROP, and APPS with the models DeepSeek-V4-Flash and Qwen-3.8-Flash, TRUST reduces the average ratio of accepted inferior updates from the greedy gate's 45.59% to 2.78%, while achieving higher final scores than the state-of-the-art statistical gate baselines for recursive self-improvement. These findings demonstrate that TRUST strikes a balance between statistical reliability and performance gain in recursive self-improvement.
Large language models (LLMs) often do not produce calibrated uncertainty estimates for their answers. We propose CAELUM, a framework that builds semantic prediction sets while separating aleatoric from epistemic uncertainty. CAELUM samples answers using an ensemble of clarification prompts, clusters them by semantic equivalence, and estimates the aleatoric and epistemic components via kernel-based von Neumann entropy before calibrating nonconformity scores with split conformal prediction. We use a weight $w_\lambda(x)=1+U_{\mathrm{ale}}(x)+\lambda\,U_{\mathrm{epi}}(x)$ to control for the contribution of the epistemic uncertainty of each instance and fit a coefficient $\lambda^\star$ to minimize prediction set size. When the objective is nonflat and selection is stable across splits, $\lambda^\star$ could be a diagnostic, where large values may indicate epistemic dominance (retrieve, abstain) and $\lambda^\star\to 0$ aleatoric dominance (clarify). For individual queries, a log ratio $R(x)$ compares the weighted epistemic component with the aleatoric component. Theoretically, we demonstrate that the coverage is capped by matchability, the fraction of items whose reference answer appears in any sampled cluster. Empirically, this matchability can be predicted from the original question features using a lightweight classifier. We use those predictions to define a stratum and derive a conformal coverage bound within it. Across 30 LLM and dataset combinations (4-32B), mean coverage is 0.74 in the learned stratum versus 0.65 overall, averaged over the 78 evaluations using the learned classifier. Project page: https://unco3892.github.io/caelum/
Standard causal discovery postulates that structure can be described by a directed acyclic graph (DAG), which can be partially reconstructed by using the faithfulness principle. According to this principle, conditional independencies in the observational distribution correspond exactly to those encoded in the unknown DAG. However, such algorithms are challenged by the fact that, without even stronger assumptions, approximate failures of faithfulness are empirically undistinguishable from exact ones. This leaves an inferential gap which can lead to major implications to causal effect identification. We introduce a different perspective, not reliant on exact independencies. By considering the specialized problem of discovering whether unmeasured confounding can be ruled out for a particular cause-effect pair, we introduce a principle we call naturalness, in which the strength of observational associations bounds the strength of causal dependencies. The bounding function is unknown, with degrees of freedom not present in faithfulness. We model uncertainty about this function using a Bayesian framing which allows for the use of experiments from separate studies, grounding an adaptive causal discovery principle on historical experimental data. We demonstrate its usefulness in partial identification of causal effects, and discuss how the promising results can motivate new principles for general causal discovery.
We propose SAMBoost (Spatially Aware Mesh Boosting), a gradient boosting method that combines tree-based covariate modelling with a stochastic partial differential equation (SPDE) latent spatial field through a single penalised objective. At each boosting iteration, the predictor is decomposed into separate covariate and spatial components. The method is motivated by applications in which the spatial effect is not a nuisance to be removed but a quantity of interest that must be recovered alongside the covariates. In insurance, for example, practitioners need both flexible nonlinear covariate effects and an interpretable geographic risk surface that can be reported on its own. Existing approaches often involve trade-offs, offering flexible covariate modelling but no explicit spatial field, rigorous spatial inference with a fixed additive covariate structure, or a combination of both at computational costs that limit practicality for large datasets. SAMBoost addresses these goals within the boosting loop, and the underlying mesh produces a sparse precision structure whose cost is largely independent of sample size. On simulations and on an open benchmark, SAMBoost matches or improves on the predictive accuracy of joint Gaussian process and tree boosting approaches. It is also faster and recovers the spatial field and covariate effects more accurately. When covariate effects include interactions, it outperforms additive Bayesian spatial models. Applied to a flood insurance portfolio with 4.9 million records, SAMBoost trains on the full training set without subsampling and runs about sixteen times faster than a Bayesian reference, with a consistent spatial risk surface. A public Python implementation is available.
LLM read-and-evaluate tasks include study screening, data extraction, risk-of-bias assessment, and critical appraisal in the evidence synthesis field. These task differ in the amount of LLM interpretation they require. Some tasks involve retrieving relatively simple information, such as an author name, sample size, or publication year. Others require the LLM to read source material, interpret it against predefined criteria, and make a judgement. Researchers should assess whether LLM judgements are accurate, supported by appropriate source evidence, and reproducible under specified conditions. A correct judgement does not necessarily indicate that the underlying evaluation process is reliable. We developed a taxonomy for identifying mechanisms that may compromise these properties and for informing the selection and reporting of appropriate evaluation methods. The taxonomy outlines potential ways that LLM judgments from read-and-evaluate tasks can fail, called TRAILS (Taxonomy of Reliability And Instability in LLM-assisted evidence Synthesis). The TRAILS taxonomy addresses two complementary questions. First, what mechanisms can cause the same read-and-evaluate task to produce different judgements across repeated LLM runs? Second, what mechanisms can systematically affect the judgement or required output in a way that is not apparent from the judgement itself? This study presents 38 mechanisms and identifies the deployment settings in which each can operate, approaches for testing them, and mitigation strategies. We also show why repeatability or accuracy does not rule out a hidden judgement failure and why human evaluation should assess whether a judgement is grounded in the supplied evidence, independent of inappropriate external influence, and produced through a process that can be examined and verified.
Mamba-2 compresses all context into a fixed-size recurrent state that dominates decoding memory and bandwidth. Existing compressors quantize state values or prune coordinates with per-channel scores. Trained states are nearly rank-one and bias-like, aligned with the mean write; but the output's energy spans an order of magnitude more directions. Per-channel scores see only the diagonal of the write--read operator, so they cannot select the subspace that governs the input--output map, which is a rotation of the coordinates. We therefore propose \method{}, which keeps that subspace by applying balanced truncation exactly via Mamba-2's scalar-identity transition. Layers differ just as sharply: most are bias-dominated, while the few that carry long-range associations are the flattest and need most of their state. We therefore introduce \spar{}, which redistributes a fixed total rank budget across layers according to the Hankel spectrum. On Mamba-2 checkpoints from 130M to 2.7B, \method{}+\spar{} halves the state at an average cost of $0.3$ WikiText-2 perplexity points, against $1.6$ for GHOST, while retaining $97\%$ of the dense model's five-digit in-context recall. At a quarter of the state, giving every layer the same rank keeps $5\%$ of that recall and \spar{} $77\%$, for a $7\%$ perplexity cost.
Knowledge graph learning offers a powerful framework for representing and inferring structured knowledge, with broad practical applications. Two obstacles limit current methods: labeled triples are scarce for each entity and relation, which makes expressive models hard to train, and scoring functions are designed ad hoc, which limits their generalizability and leaves them without theoretical grounding. We address both obstacles with a theoretically grounded framework, with end-to-end guarantees, that extends and subsumes existing methods. The framework proceeds in two stages: multi-view unsupervised pretraining that integrates heterogeneous sources of entity side information into a shared representation, followed by supervised learning across multiple relation types. We establish a unified nonasymptotic risk bound that combines the subspace recovery error of pretraining and the sample complexity of the labeled data in a single guarantee, quantifying formally how large-scale unlabeled data from multiple views benefits downstream knowledge prediction. Synthetic experiments validate each component of the theory, and experiments on large-scale real-world knowledge graph benchmarks confirm the effectiveness of our approach.
Many real-world processes can be represented as compositions of functions along a directed acyclic graph (DAG). In causal modelling, these correspond to the underlying mechanisms; in engineering, to multiple fidelity levels; and in gene-regulatory networks, to transcription factors. These functions are partially observed across the DAG, with noisy and heterogeneously sampled measurements, posing significant challenges for reconstruction, uncertainty propagation, and inference. To tackle these challenges, we place priors over functions and naturally arrive at Deep Gaussian Processes over DAGs. We theoretically study their prior-collapse behaviour and the effect of graph topology and intermediate observations on the preservation of information. We obtain almost-sure lower bounds on the asymptotic frequency of depths at which the distinction between inputs is preserved, identify broad kernel classes for which these hold, and prove an observation by \citep{dunlop2018} on the role of input connections. We offer a structured variational approximation that retains graph dependencies, preserves compositional uncertainty, and captures the explaining-away behaviour of colliders. Finally, we empirically validate our theory and methodology at scale and under DAG misspecification. We then model a latent-collider DAG, a protein signalling network, and a multi-fidelity \mbox{heavy-ion} collision emulation task, attaining state-of-the-art performance while recovering low-fidelity contributions and yielding interpretability over the simulator hierarchy.
Clustering is a central tool for discovering latent structure in unlabeled data; yet modern clustering pipelines often end with a hard assignment of each observation to a cluster without rigorous measures of assignment uncertainty. We propose a novel weighted conformal approach for constructing valid confidence sets for cluster labels. The key difficulty is that the labels available for calibration are not observed ground-truth labels, but synthetic labels produced by a data-dependent clustering algorithm. Our method develops a conformal inference algorithm that corrects the resulting mismatch with the latent target labels through weights by formulating conformal clustering as a conditional label-distribution shift problem. We first derive an oracle procedure that attains finite-sample marginal coverage and then develop a computationally tractable and implementable version using estimated conditional label probabilities and novel augmented calibration. We show that the coverage of the estimated-weight procedure depends on the estimator, giving an explicit bound on the loss relative to the nominal level. Empirical studies demonstrate that the proposed weighted approach offers improvements over the recently proposed split conformal clustering procedure in terms of informative confidence set size, especially in nonlinear and high-dimensional clustering applications.
Stability Selection finds the variables that are chosen consistently across random subsamples of the data, but it can consistently choose the wrong ones. An irrelevant variable that is strongly correlated with relevant variables can be selected in many subsamples, because resampling the observations does not change the variables themselves. We call such a variable a confounder; the term refers only to this correlation and not to causal confounding. We propose 2D Stability Selection, which also adds random noise to the design matrix, at increasing noise levels, and keeps the variables that are selected throughout. On average, the added noise acts as a ridge penalty, so each fit is a noisy elastic net. This changes the conditions under which the Lasso selects the correct variables: it can restore them when a confounder would otherwise be chosen, and it can also make selection worse. For uncorrelated relevant variables we give an exact criterion for the noise levels at which the population elastic-net irrepresentable condition holds, recovery guarantees for a single fit and for the aggregated procedure, and, for thresholds above $1/2$, a bound on the expected number of false positives (under i.i.d.\ observations; its closed form also needs exchangeable irrelevant variables). In simulations, the method excludes the confounders that Stability Selection includes. In the simulated design, an elastic net used without any noise, with a fixed ridge weight or along a deterministic path, excludes the confounders as well, which suggests that for the Lasso the ridge effect, not the randomness of the noise, is responsible.
The Super Learner (SL) is a widely used ensemble method that combines point predictions from a library of learners based on their predictive performance. Interval predictions are of considerable practical interest because they allow uncertainty in predictions produced by an individual learner or an ensemble to be quantified. Several methods have been proposed for constructing interval predictions based on the SL, however, these approaches are typically justified using asymptotic arguments or rely on computationally intensive procedures such as the bootstrap. Conformal prediction (CP) is a machine learning framework for constructing prediction intervals with finite-sample and asymptotic coverage guarantees under mild conditions. We propose coupling CP with the SL through a natural construction that mirrors the original SL framework, using individual learner weights and combining learner-specific conformal prediction sets via a weighted majority vote. We characterize the properties of the resulting SL-based prediction intervals for continuous outcomes. We cover settings under exchangeability, and data-generating mechanisms exhibiting heteroscedasticity, sparsity, and other forms of distributional heterogeneity. A comprehensive simulation study shows that the conformalized SL achieves valid finite-sample coverage with competitive performance relative to the true data-generating mechanism. We present an application of this methodology to predict creatinine levels using socio-demographic, biometric, and laboratory measurements. This example demonstrates the benefits of an ensemble with carefully selected learners designed to capture key aspects of complex regression functions, including non-linear effects, interactions, sparsity, heteroscedasticity, and robustness to outliers.
Modeling high-dimensional dependencies while keeping likelihoods tractable re- mains challenging. Classical vine copulas provide an explicit dependence structure but require repeated, costly bivariate fitting. We propose Vine Denoising Copula (VDC)1, an amortized estimator for simplified vines on continuous data. VDC trains a single bivariate denoising network on simulated copulas and reuses it across all vine edges. For each edge, the network maps an empirical histogram to a density grid, and iterative proportional fitting approximately enforces uniform grid marginals. The predicted densities and cached conditional transforms support stan- dard vine recursion, likelihood evaluation, sampling, and information estimation. This retains the classical vine factorization while replacing repeated per-edge fitting with reusable neural inference. Across synthetic and real-data benchmarks, VDC combines accurate bivariate density estimation, competitive information estimates, and faster high-dimensional vine fitting after pretraining. Performance varies across datasets; finite grid resolution and conditional prediction remain limitations. The explicit fitted structure supports edge- and tree-level information decomposition, making repeated dependence analysis more practical.