PyTorchKR Papers

최신 논문

총 817편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2611.05238v1stat.ME

Post-stratified importance sampling

Importance sampling is a building block of modern statistical computing, but the weighted estimate is prone to high or infinite variance. Moreover, its empirical standard error is often unreliable, leaving importance sampling without practical uncertainty quantification. Motivated by multilevel regression and poststratification (MRP) in survey sampling, we propose post-stratified importance sampling (PIS) for estimating target expectations from proposal draws. PIS reformulates arbitrarily high-dimensional weighting into two one-dimensional probabilistic modeling problems: regression for the conditional mean given the importance ratio and marginal density estimation for the importance ratio. Different modeling choices yield a family of PIS estimators, encompassing ordinary importance sampling as a special case. We develop a model-based implementation that combines one-dimensional Gaussian process regression for the conditional mean and score-informed heavy-tailed density estimation for the marginal law, using both the observed weights and the available density evaluations. We study the oracle gains of PIS and establish consistency under general and model-specific conditions. Simulations and applications to cross-validation demonstrate improvements in estimation accuracy and uncertainty quantification.

arXiv:2611.05225v1cs.LG

Post-Inference Model Chaining: A Problem Statement and a Model-based Bootstrap for Training End-to-End Digital Twins

End-to-end digital twins can be constructed by fitting separate stage models and connecting them after inference. We investigate the Frequentist prediction properties of a commonly used Monte Carlo simulation scheme that forwards complete predictive draws through such a chain when measured stage outputs contain independent normal errors, but the latent stage means are deterministic. Analytical two-stage examples show persistent asymptotic interval and coverage inflation and, with quadratic propagation, mean bias dependent on the parameters to be estimated. Simulations of four- and eight-stage linear-quadratic univariate chains with 7, 15 or 30 training runs demonstrate pronounced input-dependent distortions. A diagnostic using error-free latent training predictors shows that the effect is mainly not attributable to errors in variables, but to the MCSim approach even showing underestimation of coverage. We propose a model-based bootstrap: an initial chain forwards fitted means, bootstrap chains are refitted on fixed-center bootstrap data while forwarding their mean prediction, and a target-stage future error enters a studentized prediction root. Across the investigated scenarios, relative mean bias is substantially reduced, and coverage is closer to the nominal 90%. At sample size 30, the eight-stage bootstrap has mean width ratios of 1.16-1.42 relative to the known-parameter normal reference, compared with 4.11-4.97 for observed-to-observed MCSim. These results support improved endpoint prediction in the tested setting. Further investigations of the bootstrap scheme in the supplement for a bivariate 6-stage process show similar satisfactory results.

arXiv:2611.04848v1cs.AI

Amortized Nonlinear Variable Selection with Prior-Data Fitted Networks

Variable selection is an important task for interpreting potentially complex systems in fields such as biology, business, and sociology. The importance of variable selection has attracted considerable attention in statistics and machine learning. In this paper, we are mainly interested in model-agnostic variable selection for identifying variables that have general nonlinear dependence on the response. We propose MambaPFN, short for Model-Agnostic Markov BlAnket selection with Prior-data Fitted Networks, which is built upon the Transformer, a building block of many popular large language models. Specifically, we consider a functional-mapping perspective on variable-selection learning, in which the Transformer is treated as a learnable mapping from a dataset to its relevant variables. To learn this mapping, MambaPFN introduces three new components: (i) a faithful, rich, and scalable scheme for generating pretraining data, (ii) an objective that targets each variable's posterior inclusion probability, and (iii) a modified Transformer architecture tailored to variable selection. Once pretrained, MambaPFN is applied to new datasets without parameter updates. Empirically, MambaPFN achieves desirable variable-selection performance in simulation studies and exhibits a scaling law---more pretraining datasets and larger models can continuously improve its performance. Applied to a Facebook comments dataset, MambaPFN selects the fewest variables and achieves the highest computational efficiency and predictive performance, improving upon the best baselines by about 20%.

arXiv:2611.04595v1eess.SP

Directional Statistics in Signal Processing: A Geometry Driven Overview of Models, Learning, and Applications

Many signal-processing objects are defined by periodicity, normalization, sign or global-phase ambiguity, basis invariance, or positive-semidefinite constraints and therefore cannot be modeled faithfully in ordinary Euclidean space. This paper develops a geometry driven framework for directional statistics in signal processing, connecting signal representations to their natural manifolds or quotient spaces and to probability models compatible with their invariances. After reviewing instances of directional statistics in the signal processing literature, we review circular, toroidal, spherical, projective, Stiefel, Grassmann, and covariance-valued distributions, emphasizing their statistical assumptions, computational requirements, and dimensional limitations relevant to signal processing. We describe various modeling and estimation strategies, including analytical and gradient-based maximum likelihood, structured low-rank parameterizations, mixtures and hidden Markov models (HMMs), score-based and contrastive estimation, and geometry-aware K-means. To exemplify the consequences of geometry-aware modeling, we study multivariate phase coherence in synthetic data and functional neuroimaging, comparing toroidal phase vectors to projective, Grassmann, and positive-semidefinite representations. We also study speaker separation in a car environment, showing that directional representations outperform a standard Euclidean model. These examples demonstrate that when the geometric representation, statistical distribution, inference method, and computational regime are considered together, they have the potential to lead to improved models across a broad range of signal processing applications.

arXiv:2611.04157v1stat.ML

Nonparametric GARCH: An Iterated Neural Estimator

We study nonparametric estimation and forecasting for GARCH-type processes and propose an iterated neural estimator that alternates between estimating the recursion map and updating the unobserved conditional variances. Under mild conditions, we establish finite-sample bounds for the time-averaged in-sample variance-path error and show that the estimator is minimax rate-optimal up to logarithmic factors. Under stationarity and mixing conditions, we establish the same rate for recursion-map estimation and derive finite-sample bounds for out-of-sample variance forecasts and expected excess Gaussian quasi-likelihood (QLIKE) loss. The convergence rates depend on the true lag orders and remain unchanged when the fitted orders are overspecified. Simulation studies illustrate the effectiveness of our estimators under nonlinear and time-varying volatility dynamics. Real-data applications suggest that our neural methods perform competitively and can incorporate option-price and high-frequency information.

arXiv:2610.36601v1math.OC

Axiomatic Learning and Probabilistic Structures with first applications to variants of Arithmetic Machine Learning and beyond

We introduce the Axiomatic Learning Structures (ALS) and Axiomatic Probabilistic Structure (APS) in order to study the interactions of Machine Learning, Optimization, with research frameworks- which, we take the view, can be characterized at the core by the corresponding axioms. The ALS explores the interplays between structures that one is interested in Learning Theory such as Loss functions and the axioms of the objects and transformations involved in these functions. The APS studies the interaction between structures of interest in Probability Theory such as Gibbs distributions, equations with noise, etc. and the axioms of the corresponding objects and transformations. We also introduce in Section 6 Transformations-Axiomatic Based Learning (TABLe) on Mathematical data-sets. One feature of this approach is the ranking of transformations from a table for a certain task based on scores which reflect which transformations work better than others, similarly to when human investigators try various methods on a task. We also study structure recovery from noisy data. In particular, to exemplify the ideas, we consider loss functions which contain derivations and derivations-like transformations. As a first application we study recovery in the context of polynomials with integer coefficients using standard derivatives along with other types of derivations which appear in Arithmetic: the p-derivations which satisfy \textit{different axioms} from the usual derivatives. As a first application of the APS we study a class of noisy (partial/mixed) derivations equations. In the last sections, to provide further potential applications of this framework, we give an overview of an initial simplified Opinion Dynamics model and its (simultaneous) study of rules of updates.

arXiv:2609.26142v2stat.ME

하나의 해답을 향한 세 가지 경로: 응용 연구자를 위한 AIPW, TMLE 및 이중 머신러닝의 일치화

Three Routes to One Answer: Reconciling AIPW, TMLE, and Double Machine Learning for Applied Researchers

Augmented inverse probability weighting (AIPW), targeted maximum likelihood estimation (TMLE), and double/debiased machine learning (DML) are three asymptotically equivalent routes to the same efficient influence function for the average treatment effect. We contribute a worked, shared-nuisance reconciliation on real data: what a practitioner must hold fixed for the routes and their software to agree, and when they do not. Using open data on smoking cessation and weight change (n = 1566), one Super Learner library, and folds matched within schemes, we build all three by hand, in full-sample, cross-fit, and double-cross-fit variants. A semi-synthetic benchmark with known sample-average effects on a fixed design grades interval coverage. The six doubly robust estimates span 3.32--3.39 kg over repeated splits. Single runs of our code and the tmle, AIPW, DoubleML, and tmle3 packages span 3.32--3.49 kg, compatible with fitting-scheme differences and single-split noise. Over 20 seeds, the three implementations sharing our library agree within 0.01 kg in median, although one is a TMLE. Agreement can fail: with in-sample nuisance fits on a binary mortality outcome, and under poor overlap in a job-training example. We recommend diagnosing overlap before choosing an estimator, and provide a reporting checklist and software code.

arXiv:2609.09981v4stat.ML

강화학습에서의 최적 가치 추론

Optimal Value Inference for Reinforcement Learning

We study offline inference for the optimal value in reinforcement learning under finite state and action spaces. Two new nuisances are derived as fixed points of a self-induced Bellman equation, in which we approximate the maximum Bellman operator by its softmax correspondence. We propose a debiased estimator through the Neyman orthogonality and establish its asymptotic normality under diverging horizons even when the behavior policy changes with time, as long as the nuisances have the statistical rates that can be achieved by many machine learning methods. We provide a concrete estimating procedure for these nuisances and show they can lead to valid inference. Synthetic experiments validate the numerical performance of our inference method, and we implement it in real-life decision-making problems, including bike repositioning and AI agentic tool use.

arXiv:2611.03614v1cs.LG

Benchmark Prediction with Sparse and Unpaired Cached Responses

Evaluating a generative model on all queries from all available benchmark tasks is prohibitively expensive. As such, models are often evaluated on different subsets of tasks and/or different subsets of queries from a given task. The resulting collection of responses and the corresponding (model, task) score matrix are sparse and unpaired. In this paper, we introduce the Product Kernel Perspective Space (PKPS), a collection of low-dimensional representations of models based on their embedded responses. PKPS generalizes existing methods for representing black-box generative models by comparing responses to similar -- rather than just identical -- queries and is therefore able to utilize sparse and unpaired information. Theoretically, we show that benchmark prediction using PKPS is query-efficient relative to methods that only use information from dense and paired portions of the evaluation data. Empirically, we show that PKPS-based methods outperform baselines on benchmark prediction across two large benchmark suites.

arXiv:2611.03166v1stat.ML

Causal discovery needs explicit epistemic standards

There are many causal discovery algorithms, but few guiding principles for choosing between them. Methods are usually compared through three lenses: identifiability, consistency, and benchmarks. Each introduces assumptions about the data-generating process that are often untestable. We argue that there are no shared standards for comparing, prioritizing, or even discussing those assumptions. We support this claim by showing that (i) identifiability assumptions are often preferred by convention rather than by principle, (ii) uniform consistency requires further untestable assumptions, while the weaker pointwise consistency has arguably narrowed algorithm design, and (iii) benchmarks are not neutral tests but carry an epistemic role of their own: they quietly fix which regimes matter, which guarantees come into play, and what counts as success. In doing so, they shape what algorithm comparisons are taken to show and thus perform some of the same epistemic work as theory, but less transparently. In this position paper we argue for a more transparent, integrated practice: Make the epistemic standards for discussing identifiability and consistency assumptions explicit, especially their failure modes and scope conditions; design benchmarks to operationalize those standards, by stress-testing and ablating both. Then, theoretical and empirical results jointly clarify when, why, and how algorithms may work rather than accumulate as disconnected advances.

arXiv:2611.02639v1cs.LG

Where Is a Prediction Model Miscalibrated, and for Whom? Local Audits in Outcome-Learned Pockets

Overall and group-level averages, and ranking metrics such as the AUC, can hide sets of observations for which a prediction model systematically over- or under-predicts. We present an audit that asks where a model is miscalibrated (in which sets of similar observations it over- or under-predicts on average) and for whom (whether its average prediction error differs between demographic groups, such as by race or sex, within the same set). It groups observations into pockets using random-forest similarity, which weighs each feature by its importance for predicting the outcome, with a ridge-regression grouping alongside. Pockets are learned on one sample and tested on another, with simultaneous confidence intervals that allow for correlation within units such as hospitals; the audit needs only features, outcomes and fixed predictions. In 17 simulation scenarios with an error planted for one group, the combined audit located it better than the strongest alternative grouping in 15, with false-alarm rates near or below 5%; with a local error planted for everyone, it won in 12 of 15 scenarios at the smaller error size and lost when the error was unrelated to the outcome. In public synthetic data from the care-management algorithm studied by Obermeyer et al., it found no detectable overall error but under-prediction of illness in the sickest pockets and over-prediction elsewhere, and distinguished Black patients' concentration where the score under-predicted for everyone from a Black-White gap within pockets. For APACHE IVa, an exploratory analysis located its previously reported over-prediction of ICU mortality in a mostly ventilated pocket and a low-risk pocket, with none detected in a moderate-risk one. Two chest X-ray classifiers showed offsetting local errors behind no detectable overall error. Code: https://github.com/modenesi/pocket-audit

arXiv:2611.02432v1cs.CL

Diagnosing Model--Item Interactions in Language Model Benchmarks with LSIRM

Language models with similar benchmark accuracy can succeed on different items. Overall scores do not show where their performance differs or which item properties might explain those differences. We demonstrate how the latent space item response model (LSIRM) can help evaluators investigate these patterns and their consequences for model comparison. We analyze 1,348 language models on 5,761 BBH items and compare ways of presenting and scoring answers across BBH, GPQA, MMLU-Pro, and MuSR. We examine LSIRM maps, which summarize differences in performance across models and items, alongside item properties and records of correct and incorrect responses. In BBH, nearby items on these maps often share the same correct answer, such as "Yes" or "No." Across the four benchmarks, this pattern is stronger when answers are scored using option labels, such as A, B, C, and D, than when using the text of the options. We then estimate each model's tendency to perform better or worse on items with particular correct answers and quantify how this tendency contributes to its score. This separates an estimated contribution associated with correct answers from the remaining differences in item performance. Two models' total numbers of correct answers differed by only 14 out of 5,761 items. Excluding the estimated answer-category contribution from their scores moved their ranks hundreds of places in opposite directions, while keeping all other estimates unchanged. The approach helps evaluators identify differences hidden by overall accuracy and understand how the choice of what to include in a score affects model rankings.

arXiv:2611.02413v1cs.LG

A Scientific Claim Stability Framework for Evaluating MRI Morphometry Pipelines

MRI morphometry pipelines are often evaluated by measurement agreement or downstream accuracy, yet their scientific value depends on the conclusions they support. Two pipelines can produce similar feature-level results yet support different conclusions about aging, cognitive decline, or neurodegeneration. We introduce Scientific Claim Stability (SCS), a claim-centric evaluation framework that converts region-of-interest (ROI) morphometry tables into auditable claim cards and evaluates their support under changes in dataset, processing pipeline, atlas, feature family, and target. Across 10,370 MRI scans from three datasets and five cohorts, SCS generates 2,783 claim cards and evaluates 89 claim-family pairs. The evaluation distinguishes stable, transferable, and fragile claims. FreeSurfer 8-FastSurfer comparisons show high evidence concordance and low anatomical-support overlap under the primary covariate specification. Acquisition-adjusted analyses show that the magnitude of support disagreement depends on the model specification. Structured review by specialists in geriatrics, neuroscience, and radiology, alongside auxiliary ChatGPT and Gemini reviewers, assesses claim plausibility, clarity, faithfulness, usefulness, and caution. SCS provides a reproducible way to audit the evidential stability of morphometry claims without assuming a universal gold standard.

arXiv:2611.02254v1stat.ME

Partial Invariance for Classification under Structural Causal Models

Making predictions in an unseen environment from data collected across multiple training environments is challenging. This challenge is particularly pronounced when the target-generating mechanism varies across environments. In this work, we introduce environment-label separation, a form of partial environmental invariance that decomposes the target conditional distribution into an environment-invariant component and a label-invariant component. We show that this partial invariance enables estimation of the target conditional distribution in an unseen environment when using labeled training data together with unlabeled test features. Within the structural causal model framework, we provide sufficient conditions for environment-label separation. Building on these conditions, we develop an algorithm, dIMP-Classifier, which jointly minimizes an estimate of the test risk and encourages invariance across environments. We further provide finite-sample guarantees that characterize the test risk in terms of deviations from invariance, estimation error, and sample complexity. Experiments on simulated data and the real-world light tunnel Causal Chamber data demonstrate that the proposed approach can perform under interventions on the target mechanism and consistently outperforms the considered standard and invariance-based baselines.

arXiv:2611.02122v1stat.ME

Optimal spectral shrinkage via Krylov approximation

Spectral shrinkage encompasses a broad family of estimators for high-dimensional linear regression that regularize estimation and prediction by applying spectral filters to the sample covariance matrix. Despite its widespread use, achieving oracle-optimal prediction risk under general covariance structures remains challenging, as the oracle filter requires unknown population quantities and has a closed form only for specific covariance models such as isotropic and spiked models. We propose \textit{risk-targeted Krylov fitting} (RTK), a data-driven spectral estimator that adapts to the unknown covariance structure and attains the oracle risk asymptotically. Like partial least squares, RTK operates in an $m$-dimensional Krylov space, but it targets the oracle risk directly by minimizing the leave-one-out risk, which we compute exactly without refitting the model $n$ times. Under general distributions and covariance structures, RTK attains the optimal prediction risk in the Krylov space up to $O(1/n)$. Furthermore, its risk approaches that of the optimal analytic filter with $m$ of order $\log n$. In simulations over a wide range of covariance structures, RTK and a truncated variant were robust to the choice of $m$ and consistently among the spectral estimators closest to the oracle. In applications to single-cell sequencing data and large-scale neural recordings, the proposed methods achieved higher prediction accuracy than competing estimators on most tasks.

arXiv:2611.01566v1stat.ML

Principled Human-AI Collaboration in Agentic Workflows

Organizations adopting AI face the challenge of deciding which outputs can be released without human review while meeting the quality standards of human experts. We develop a statistical framework for delegating tasks between experts and black-box generate-verify workflows, in which AI-generated answers are verified and revised over multiple rounds. Before deployment, human experts evaluate selected outputs, and their judgments are used to calibrate the verifier threshold and number of attempts. Our method selects the policy with the highest estimated automation rate among statistically approved candidates, without modifying the generator or verifier. We establish finite-sample control of the error rate among released outputs, characterize the expert evaluations sufficient to attain a target automation rate, and derive a minimax lower bound establishing rate optimality in the quality margin and required confidence. In a MultiHiertt financial question-answering case study, average held-out automation increases from 2.50% to 24.17% as expert evaluations accumulate, while released-output error rates remain around 1.20%. In contrast, heuristics that release answers after consistently high verifier scores either violate the 5% error tolerance or automate at most 1.30% of tasks. Synthetic experiments further examine these theoretical predictions under controlled settings.

arXiv:2611.01436v1stat.ME

Transport for Conditional Sampling: Foundations and Applications

Conditional sampling is central to Bayesian inference, generative modeling, and causal inference: given a joint law for random variables $X$ and $Y$ and an observation $Y=y$, one seeks to sample from the conditional law of $X\mid Y=y$. Even when the joint distribution is accurately represented, this task can be computationally difficult as it may require evaluating intractable normalizing constants or solving a new inference problem for each observation. Measure transport provides an approach to this problem by identifying a map with triangular structure that directly pushes forward a reference to the conditional of a joint distribution for any observation. This article reviews the theoretical, methodological, and algorithmic foundations of triangular transports, with the Knothe-Rosenblatt rearrangement as a canonical construction. We discuss methods for estimating triangular maps from finite samples, as well as their use in simulation-based inference and causal inference. While our primary aim is to review existing formulations, we also present new ideas and open questions for conditional sampling using transport.

arXiv:2611.01312v1stat.ME

Accuracy Is Not Risk: Why Medical AI Should Not Report Only Its Most Likely Label

Background. Many medical AI models are trained to reproduce a single reference label and hand the most likely label to the next step in a clinical workflow. A classifier can select that label correctly while providing an inadequate summary for estimating downstream risk. Methods. We use Jeffrey's rule of conditioning to specify when probabilities over intermediate states, such as diagnoses or imaging categories, can be carried forward to an outcome risk. We separate two errors of reporting only the most likely label, discarding the probability of the other states and applying the risk of patients who have the leading diagnosis to a patient who only probably has it. We also give conditions under which the label and the full distribution lead to the same decision, and we propose guardrails for device developers, regulatory reviewers, and health insurers. Results. In a hypothetical chest-pain differential, reporting only the benign leading diagnosis greatly understates the patient's risk of death or cardiac arrest, so a patient who warrants evaluation could be discharged or denied imaging. In a hypothetical BI-RADS example, the most likely category is "probably benign," yet the propagated malignancy risk exceeds the ceiling that defines that category. Conclusions. A predicted label is not an observed state, and a calibrated classifier is not automatically a valid outcome model. Clinical and insurance coverage decisions should rest on the patient's outcome risk, judged against a threshold justified by the consequences of acting and not acting. Developers should state what each model output means and check that every step between the model and the decision preserves the patient's risk.

arXiv:2611.01134v1stat.ME

Selective inference for pruned interaction trees with known unequal variances in randomized trials

Interaction trees reuse outcomes to discover treatment-effect heterogeneity, complicating inference after growth and pruning. We develop selective inference for generalized least-squares (GLS) interaction trees with a common covariate basis and known unequal variances. Under a conditional Gaussian model with positive diagonal covariance, weighted orthogonality reduces survival under additive fixed-penalty pruning to one quadratic inequality supplementing the growth event. This yields exact finite-sample conditional tests and confidence intervals, with retained-path and complete-tree inference agreeing on compatible sections. Under causal identification and a correct reference prognosis, the pruning-invariant GLS target is a signed finite-design weighted contrast of conditional average treatment effects. Simulations at $n=100,200,400,800$ and maximum depths one to three search all adjacent-observation midpoints, support calibration and demonstrate over-rejection when pruning survival is omitted. Compared with a 50:50 honest procedure, full-data learning improves reference-split recovery in some small-sample settings. Root-level median intervals can be shorter, while deep-node intervals and upper tails can be much wider; these comparisons concern each procedure's selected targets. Estimating a common scale with the variance ratio fixed produces modest changes in Selective coverage in the settings examined, without extending the exact guarantee. An exploratory ACTG 175 analysis provides a working-model illustration with estimated arm variances.

arXiv:2611.00979v1math.ST

Optimal Learning in Fairness-Aware Contextual Bandits

Mitigating unfair disparities in sequential decision-making is a growing concern in machine learning. We study contextual bandits under a global covariate-based Lipschitz fairness constraint, pioneered by \cite{dwork2012fairness}, requiring individuals with similar contexts to receive similar action distributions. We show that this constraint induces a randomisation region near the decision boundary and explicitly quantify the population level cost of fairness. We further establish a canonical representation of the optimal fair policy, which reduces the fair policy design to a lower-dimensional boundary optimisation problem. We develop a lazy fair greedy algorithm that exploits fairness-induced covariate diversity under continuous contexts, with policy updates computed via finite-dimensional linear programming. For general context distributions, including discrete and mixed contexts, we propose a lazy polyhedral upper confidence bound (UCB) algorithm, whose optimistic optimisation reduces to finitely many linear programs over confidence-set vertices. Both methods satisfy fairness at every round and achieve regret of order $O(\sqrt{T} + T^{(d-2)/(d-1)}{\indc\{d\geq 4\}})$ up to poly-logarithmic factors, where $d$ is the dimension of the contexts. A matching minimax lower bound for $\ell_1$-based fairness establishes optimality, and reveals an unavoidable nonparametric cost of learning the decision boundary, which dominates the parametric learning cost when $d \geq 4$. Experiments on synthetic and real data support our theoretical findings and demonstrate the practical effectiveness of our methods.