PyTorchKR Papers

최신 논문

총 817편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2611.07937v1stat.ML

Joint Modeling of Conditional Mean and Variability in Multilevel Functional Data: Application to Postprandial Glucose Trajectories in the AEGIS Study

Functional data analysis provides a framework for studying time-varying associations between covariates and high-frequency biological signals. In digital health, participants often contribute repeated functional trajectories, yet existing multilevel regression methods largely focus on the conditional mean, overlooking variability and its role in biological heterogeneity. We propose a heteroskedastic functional mixed-effects regression framework that jointly models the conditional mean and variance of repeated trajectories, with participant-specific random effects in both components. We also develop a participant-level bootstrap to quantify uncertainty in estimated functional parameters. We apply the method to postprandial glucose trajectories from the AEGIS study. Premeal glucose, carbohydrate intake, and meal type show distinct time-dependent associations with glucose variability, whereas HbA1c is more strongly associated with the conditional mean than with variability. Substantial between-participant differences in residual variability remain even after accounting for observed covariates. These findings show that modeling conditional variability alongside the mean provides insights into glucose regulation and individual heterogeneity that mean-focused analyses may miss. More broadly, the proposed framework enables joint characterization of average responses, variability, and participant-specific heterogeneity in repeated biological signals.

arXiv:2611.08077v1stat.ME

Doubly-robust causal inference for multi-valued treatment regimes with recurrent neural networks

We propose a doubly-robust method to estimate the causal effects of multi-valued treatment regimes on patient outcomes. A multi-valued treatment regime assigns personalized treatments from a set of competing options based on a patient's history. Existing methods for optimal treatment regimes are largely limited to binary or continuous treatments. We extend the longitudinal targeted minimum loss-based estimator (LTMLE) to accommodate multi-valued treatments by modeling treatment assignments with a multinomial model. We further integrate recurrent neural networks (RNNs) into the LTMLE framework as learners for the outcome, treatment, and censoring models, and compare this approach to ensemble machine learning. Simulation studies show that LTMLE with either ensemble machine learning or RNNs is approximately unbiased, with close to nominal coverage, whereas sequential g-computation with ensemble machine learning is substantially biased under a dynamic rule. A multinomial treatment model performs similarly to a series of binomial regressions with ensemble machine learning, but yields more stable inference when the treatment model is an RNN.

arXiv:2611.07536v1stat.ME

Identification and Bounding of Joint Expectation over Potential Outcomes

The joint expectation of functions of potential outcomes $(Y_1,Y_0)$ is a fundamental quantity in causal inference, particularly when investigating the variation or heterogeneity of causal effects. This paper provides novel assumptions for their identification: comonotonicity and countercomonotonicity. These assumptions offer a unified framework for their identification across discrete, continuous, and mixed outcome variables. In the absence of these assumptions, we derive sharp bounds for two broad classes of functions, yielding new results for bounding moments of individual causal effects, which are central to measuring effect heterogeneity. We present corresponding estimation methods and illustrate them in simulation studies and real-world datasets.

arXiv:2611.07490v1stat.ME

Statistical and Privacy Guarantees for Deep M-Regression via Stochastic Gradient Langevin Dynamics

In this article, we study deep nonparametric regression trained via stochastic gradient Langevin dynamics (SGLD), a variant of stochastic gradient descent that injects carefully calibrated Gaussian noise during optimization. Using a PAC-Bayesian analysis tailored to deep ReLU networks, we establish an excess-risk bound that simultaneously accounts for network approximation error, statistical complexity, and the algorithmic cost accumulated along the SGLD trajectory. Under a Bernstein condition and within a stable SGLD regime, the excess-risk bound contains a stochastic term whose coefficient is inversely proportional to the variance of injected noise, while the estimator retains the optimal rate for nonparametric regression under the H\"older smoothness condition up to logarithmic factors. This result provides an approximation-error-aware, rate-optimal statistical perspective on the widely observed generalization benefits of moderate SGLD noise. For per-sample clipped SGLD, we further establish an exact f-DP guarantee and the corresponding asymptotic $\mu$-GDP guarantee for the proposed estimator. We establish statistical convergence rates and differential privacy guarantees for DNN estimators trained using SGLD, while explicitly accounting for both network approximation error and the effect of injected noise in the excess-risk bound. Extensive simulation studies and analyses of two real datasets examine the finite-sample predictive performance and privacy-utility behavior of the proposed procedure.

arXiv:2611.07399v1stat.ME

Changepoint localization via nested exchangeability tests: from point estimates to confidence intervals

We study distribution-free uncertainty quantification for a changepoint estimate in an offline setting. We observe a sequence with a single changepoint and are given an estimate of its location. The pre- and post-change segments are independent and each consists of i.i.d. (or, more generally, exchangeable) observations from arbitrary and unknown distributions. We introduce Changepoint LOcalization via Nested Exchangeability testing (CLONE), a novel method for constructing distribution-free confidence intervals around that estimate by testing exchangeability in nested windows centered at the reference estimate. Under these assumptions, CLONE provides finite-sample coverage for any black-box reference estimator. Using a geometrically shrinking radius grid, CLONE requires only $O(log n)$ permutation tests. We characterize optimal scores for the framework and provide practical recommendations that result in sharp confidence intervals. Under mild assumptions, CLONE confidence intervals have asymptotically vanishing normalized length. Simulations and experiments on image and text data support our theoretical findings.

arXiv:2609.30565v2stat.ME

High-dimensional Gaussian Graphical Model Testing for Long-Memory Time Series

Many real-world high-dimensional time series exhibit long-memory, but Gaussian graphical model testing in this regime remains understudied. We develop a direct, data-adaptive test statistic for assessing conditional independence in the graph structure of stationary Gaussian time series. We establish a finite-sample, Berry--Esseen type Gaussian approximation bound for the statistic, which applies to both short-memory and long-memory time series. The testing procedure uses a block bootstrap method, on which we provide a finite-sample validity result including in the ultra-high-dimensional scenario, and can be extended to comparing graphical structures in two-sample tests. We also develop a consistency-empowered correction to the statistic and show that such tests attain asymptotic consistency in both size and power. Our proposed method is applied to real-world fMRI data to understand functional connectivities within brain in different periods.

arXiv:2609.20260v2econ.EM

Counterexamples and Sufficient Conditions: Comments on "Optimally-Transported Generalized Method of Moments"

We comment on the optimally-transported generalized method of moments (OTGMM) of Schennach and Starck (2026) and give counterexamples to Theorems 2-6. In the small-error analysis, the assumptions do not ensure consistency or asymptotic normality (Theorems 2-3). In the large-error analysis, Theorem 4 states the equivalence between the OTGMM estimator and the GMM estimator with modified moments, and Theorems 5-6 assert consistency and asymptotic normality. Through counterexamples, we show that the equivalence in Theorem 4 fails and a component of the Lagrange multiplier has different limits under the two estimators (Theorems 5-6). We also point out that, under misspecification, the population value selected by OTGMM depends on the metric. We give sufficient conditions and establish results following the original motivation for OTGMM. Our corrections to Theorems 4-6 change their assumptions or interpretation more substantially than those to Theorems 2 and 3. Furthermore, we correct Assumption 16 and the test's null distribution.

arXiv:2605.08069v2stat.ME

Empirical Bayes Rebiasing

We study methods for simultaneous analysis of many noisy and biased estimates, each paired with an even noisier estimate of its own bias. The analyst's goal is to construct short calibrated intervals for each parameter. The standard debiasing approach, which subtracts the bias estimate from each biased estimate, inflates variance and yields long intervals. In this paper, we propose an empirical Bayes rebiasing strategy that starts from the fully debiased estimates and learns from data how much bias to reintroduce by estimating the unknown bias distribution. We provide convergence rates for the coverage of our intervals when the bias distribution is estimated using nonparametric maximum likelihood. Furthermore, we demonstrate substantial precision gains in prediction-powered inference, including pairwise LLM win-rate evaluations, as well as for inference of direct genetic effects in family-based GWAS.

arXiv:2604.26942v2cs.LG

Hyper Input Convex Neural Networks for Shape Constrained Learning and Optimal Transport

We introduce Hyper Input Convex Neural Networks (HyCNNs), a novel neural network architecture designed for learning convex functions. HyCNNs combine the principles of Maxout networks with input convex neural networks (ICNNs) to create a neural network that is always convex in the input, theoretically capable of leveraging depth, and performs reliably when trained at scale compared to ICNNs. Concretely, we prove that HyCNNs require exponentially fewer parameters than ICNNs to approximate quadratic functions up to a given precision. Throughout a series of synthetic experiments, we demonstrate that HyCNNs outperform existing ICNNs and MLPs in terms of predictive performance for convex regression and interpolation tasks. We further apply HyCNNs to learn high-dimensional optimal transport maps for synthetic examples and for single-cell data, where they oftentimes outperform ICNN-based neural optimal transport methods and other baselines across a wide range of setting.

arXiv:2511.03087v3stat.ME

Beyond Maximum Likelihood: Variational Inequality Estimation for Generalized Linear Models

Generalized linear models (GLMs) are fundamental tools for statistical modeling, with maximum likelihood estimation (MLE) serving as the classical approach for parameter inference. While MLE performs well for canonical GLMs, it can become computationally challenging in more general settings with non-canonical, non-smooth, or saturating link functions, where the resulting optimization landscape may be ill-conditioned, non-convex, or non-differentiable. In this paper, we study an alternative estimation framework based on variational inequalities (VIs), which formulates GLM estimation through an operator-based equilibrium condition rather than likelihood minimization. We analyze the VI estimator from a statistical perspective and establish finite-sample error bounds and asymptotic normality, together with convergence guarantees for fixed-point and stochastic approximation algorithms. The framework accommodates a broad class of link functions, including non-canonical, non-smooth, and non-monotone cases satisfying a strong Minty-type condition, and extends naturally to generalized additive models (GAMs) via basis expansion. Numerical experiments demonstrate that under a fixed iteration budget, the VI approach achieves competitive estimation accuracy and faster, more stable convergence than MLE, particularly in GLMs and GAMs with non-canonical or non-smooth link functions.

arXiv:2611.05966v1stat.ME

Prior-data fitted networks for variable selection

Prior-data fitted networks (PFNs) have become state-of-the-art for tabular prediction but remain underexplored in other areas of statistics. In this work, we present two PFNs for Bayesian variable selection: CD-PFN identifies conditional dependencies, and PC-PFN identifies parents and children in a causal graph. Both networks estimate posterior inclusion probabilities (PIPs) without posterior samples or data-specific modeling. In semi-synthetic and real data experiments, CD-PFN and PC-PFN often outperform existing methods, demonstrating the value of PFNs for variable selection and their potential in statistics more broadly. Open-source software is available.

arXiv:2611.05723v1stat.ME

Balance-Guided Forest for Causal Effect Estimation

Covariate balancing is widely used for estimating causal effects in observational studies. A central challenge is determining which covariate features should be balanced. Forest-kernel balancing provides a data-driven way to learn informative balancing features. Existing forest-kernel balancing methods first learn a tree-based representation and then construct balancing weights. However, this procedure creates a mismatch, as limited tree capacity may be spent on features that are already well represented. We propose Balance-Guided Forest (BGF), which uses the current balancing weights to guide subsequent tree construction. BGF grows trees using a novel splitting score that combines residual imbalance with an outcome contrast, favoring splits that separate outcome-relevant regions where imbalance remains. This couples representation learning with covariate balancing rather than treating balancing as a downstream step. We establish an oracle-type bound that links the iterative construction to weighting error and causal bias. Simulations exhibit improvements in causal estimation relative to existing forest-kernel approaches. Real-data applications illustrate the practical behavior of BGF across observational studies.

arXiv:2610.12702v2stat.ME

Product Monte Carlo: Expectation Under a Product of Densities

Aggregating independently computed sub-sample inferences is a well-known challenge in large-scale Bayesian computation, federated learning, and opinion pooling. The core problem is to approximate a distribution whose density is proportional to a product of individual densities. We propose Product Monte Carlo (PMC), an algorithm for computing arbitrary expectations under a product of densities using only Monte Carlo draws from individual densities. We turn the intractable product expectation into a regression problem by expressing it through a family of computable conditional expectations. Besides the point estimate, our fully Bayesian regression formulation provides coherent uncertainty quantification and principled diagnostics. We establish theoretical guarantees on consistency and characterize convergence rates for the PMC estimator. Across simulated and real-data experiments varying the number, dimensionality, and heterogeneity of the component distributions, PMC achieves higher estimation accuracy, better-calibrated uncertainty coverage, and improved scalability at a lower computational cost than existing methods. In addition, PMC requires no density evaluation, making it applicable to simulation-based inference, as we demonstrate in an application with protein images.

arXiv:2609.27179v2math.ST

Optimal distribution-free sequential change detection with conformal martingales

We study the challenging problem of distribution-free sequential changepoint detection for independent observations with unknown and \emph{completely unrestricted} pre- and post-change laws. For over two decades, the primary existing solution for this problem has employed Vovk's conformal test martingales and associated e-detectors, which control the probability of false alarm (PFA) and the average run length (ARL) respectively. The majority of these works focus on validity, with statistical efficiency usually left for simulations. We develop a comprehensive theory of how conformal p-values behave under non-exchangeable data with a changepoint at an unknown time $T$. We use this to analyze the post-change growth and resulting detection delays and find that the standard existing methods are provably suboptimal for PFA and ARL control, and can lead to delays that are $\Omega(T)$ and $\Omega(\sqrt{\text{ARL}})$ respectively. We construct new conformal e-processes and e-detectors with learned scores and a fixed, distribution-independent betting dictionary. For any distribution pair with positive finite KL divergence, our methods asymptotically achieve $\log T/\mathrm{KL}$ delay for PFA control and $\log\mathrm{ARL}/\mathrm{KL}$ delay (with a sharp leading constant of 1), which are both proven to be minimax optimal in a suitable double-array framework. Simulations illustrate the large empirical benefits of our new approach.

arXiv:2606.16952v3cs.LG

팬텀과 정보 노출: 합성 데이터 프라이버시 감사를 위한 통계적 프레임워크

Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data

생성형 AI와 대규모 언어 모델(LLM)의 급속한 도입은 민감한 실세계 데이터셋에 대한 프라이버시 보존 대안으로서 합성 데이터에 대한 관심을 촉발했다. 그러나 높은 유틸리티의 합성 데이터를 생성하는 것은 종종 학습 코퍼스의 사적 정보를 암기하고 그대로 재현할 위험을 수반한다. 본 연구에서는 이러한 데이터 공개를 탐지하고 설명하기 위해 설계된 맞춤형 경험적 감사 프레임워크를 제시한다. 우리 프레임워크는 시스템이 사용자 정보를 직접 재현하는 "진정한 공개(true disclosures)"와 시스템이 우연히 사용자 데이터를 생성하는 "팬텀 공개(phantom disclosures)"를 구별하는 메커니즘을 도입한다. 입력 데이터를 학습 집합과 홀드아웃 집합으로 분할하고 엄격한 통계적 가설 검정을 적용함으로써, 관측된 공개가 제로 학습(zero-learning)이나 특정 차등 프라이버시(DP) 경계와 같은 엄격한 프라이버시 기준선과 일치하는지 판단한다. 중요하게도, 이 접근법은 모델 접근, 카나리 삽입, 참조 모델 학습이 필요 없으며—합성 출력과 홀드아웃된 대조 집합만 필요하다. 우리는 이 프레임워크가 멤버십 추론(membership inference) 공격으로 효과적으로 작동하며, 기존 데이터 기반 감사 방법보다 더 엄밀한 프라이버시 유출에 대한 경험적 하한을 제공함을 보여준다. 우리 접근법은 모델에 독립적이며, 모든 합성 데이터 생성 메커니즘에 적용 가능하고, 섀도우 모델(shadow-model)이나 카나리 기반 대안보다 수 차수만큼 적은 계산 자원을 필요로 한다.

arXiv:2606.06233v2stat.ML

Anchor PCA

Principal component analysis (PCA) is one of the most widely used unsupervised dimension reduction techniques. We study PCA for data from multiple related domains. Since principal components generally differ across domains, one way to obtain a shared low-rank embedding is to perform PCA on the pooled data. However, this approach can focus on spurious directions that exhibit high variation in only a few domains. To find a robust embedding that still explains most variance in unseen but similar domains, we propose instead to focus on shared directions of variation. To this end, we introduce Anchor PCA which trades off overall explained variance with agreement between the shared and domain-specific low-rank embeddings. Anchor PCA amounts to PCA on a modified target matrix and thus can be solved efficiently. Moreover, we show that Anchor PCA recovers a maximal invariant subspace and admits a minimax reconstruction interpretation under bounded domain-specific covariance inflations. On simulated and real-world data with temporal distribution shift, we demonstrate, respectively, that Anchor PCA recovers the maximal invariant subspace and yields embeddings that explain more variance on unseen domains than the pooling baseline and a worst-case alternative. Taken together, these findings establish Anchor PCA as a promising approach to robust unsupervised dimension reduction from multi-domain data.

arXiv:2605.16828v2stat.ML

Prediction-Intervention Games and Invariant Sets

We consider the following two-player game: using observational data, the leader chooses a prediction function for a response variable $Y$ from given covariates. The follower then reacts with an intervention on some covariates in the underlying structural causal model to maximize their own objective. The leader knows the intervention targets, but may have limited knowledge of the follower's objective. We call this setup a prediction-intervention game, a special case of a Stackelberg game. Finding an optimal strategy for the leader is generally difficult. To avoid severe performance loss, the leader may base their prediction on the causal parents of $Y$, or more generally on an invariant subset of covariates. We prove, for two common classes of follower objectives, that predictors based on the stable blanket, an invariant subset, are at least as good as those based on the causal parents. We further upper bound the leader's post-intervention risk by a worst-case risk over allowed interventions and strengthen existing distribution generalization results to analyze this bound: we give sufficient conditions under which stable-blanket predictors are worst-case optimal, and show by examples that these conditions cannot in general be dropped. Finally, we discuss practical strategies for settings with known and unknown graph, and test them on simulated and real-world data.

arXiv:2603.08393v3stat.ME

Decoupling Distance and Networks: Hybrid Graph Attention-Geostatistical Methods for Spatio-temporal Risk Mapping

Accurate spatial prediction and rigorous uncertainty quantification are central to modern spatial epidemiology and environmental risk analysis. We introduce a statistically principled hybrid modelling framework that integrates the nonlinear, attention-based representation learning capabilities of a dynamic Graph Attention Network (GATv2) with a latent Gaussian spatial process from model-based geostatistics (MBG). This framework jointly captures relational dependence encoded in graph structures and continuous spatial dependence governed by physical proximity. We evaluate the proposed model via a controlled simulation study and an applied analysis of malaria prevalence data, comparing its predictive accuracy, calibration, and uncertainty quantification against classical geostatistical models and standalone GATv2 architectures. Our analyses show that GATv2 captures complex nonlinear interactions but fails to account for residual spatial autocorrelation, resulting in miscalibrated predictive distributions. Conversely, geostatistical models provide coherent uncertainty quantification through structured covariance functions yet are constrained by linear predictor assumptions and by their reliance on Euclidean distance to encode spatial structure. By integrating attention mechanisms and nonlinear features with an explicit probabilistic spatial random field, the hybrid model captured the relational dependence, consistently improved predictive accuracy, and provided more realistic uncertainty quantification in both simulation and applied settings. Overall, the findings demonstrate that the hybrid model constitutes a statistically coherent and empirically robust framework for modelling complex spatial and spatio-temporal processes in settings where both distance-based and structure-based dependencies operate.

arXiv:2507.07359v2cs.LG

Goal-Oriented Sequential Bayesian Experimental Design for Causal Learning

We present GO-CBED, a goal-oriented Bayesian framework for sequential causal experimental design. Unlike conventional approaches that select interventions aimed at inferring the full causal model, GO-CBED directly maximizes the expected information gain (EIG) on user-specified causal quantities of interest, enabling more targeted and efficient experimentation. The framework is both non-myopic, optimizing over entire intervention sequences, and goal-oriented, targeting only model aspects relevant to the causal query. To address the intractability of exact EIG computation, we introduce a variational lower bound estimator, optimized jointly through a transformer-based policy network and normalizing flow-based variational posteriors. The resulting policy enables real-time decision-making via an amortized network. We demonstrate that GO-CBED consistently outperforms existing baselines across various causal reasoning and discovery tasks-including synthetic structural causal models and semi-synthetic gene regulatory networks-particularly in settings with limited experimental budgets and complex causal mechanisms. Our results highlight the benefits of aligning experimental design objectives with specific research goals and of forward-looking sequential planning.

arXiv:2307.04754v4cs.LG

Dynamic Strategy Selection

We consider the problem of dynamically selecting among competing strategies using observable predictors. To mitigate estimation noise and excessive switching, we regularise the estimated policy with a switching cost, which may exceed actual transaction costs when these are too small to provide sufficient regularisation. Since switching costs make current choices affect future decisions, we formulate the problem using dynamic programming. The optimal policy is estimated using a reinforcement learning algorithm based on iterated regressions, making the procedure simple to implement. We apply the method to monthly rotation among momentum, value, profitability, and defensive U.S. equity factors over a sample of about 60 years. Using a parsimonious set of state variables, the estimated policy outperforms both the myopic dynamic policy and the best fixed factor in hindsight in out-of-sample evaluation. Under weak conditions, we show theoretically that the decision rule estimated from empirical data is asymptotically consistent for the unknown optimal decision rule. Simulation results illustrate the finite-sample relevance of these asymptotic results.