PyTorchKR Papers

최신 논문

총 842편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 카테고리 구독

arXiv:2611.15698v1stat.ML

Learning Mixtures of Exponential Family

Motivated by causal inference and physical computing systems, we consider the problem of learning mixture of two bounded-support exponential-family distributions with a shared interaction parameter matrix and mixture component-specific linear parameter vectors. The unknown sequence of component labels belongs to a known structured class. We propose an estimator based on a convex loss that avoids the log-partition function; for each fixed label assignment, estimation is a convex optimization problem. We show that the conditional population loss uniquely identifies the interaction matrix and the linear parameter attached to each sample. For a global empirical minimizer, we establish finite-sample bounds on the interaction error in Frobenius norm and the average squared linear-parameter error. The bounds depend on the logarithm of the number of admissible label assignments, and, for fixed dimension and model bounds, imply consistency when this complexity grows sublinearly with sample size. Additional occupancy and separation conditions yield component and label recovery.

arXiv:2611.15634v1stat.CO

Sampling the Funnel: A Comprehensive Survey of Hamiltonian Monte Carlo for Hierarchical Bayesian Inference

Hierarchical models are central to applied Bayesian statistics, yet the variance structures they depend on can produce posterior geometries that remain difficult to explore even with Hamiltonian Monte Carlo (HMC), a designation now covering a large and increasingly heterogeneous family of algorithms. This survey uses Generalized HMC as an organizing framework to describe the many variants of HMC, their methodological and computational properties, and their empirical consequences. Varying the trajectory length, the degree of momentum refreshment, and the presence of a Metropolis correction recovers several familiar Hamiltonian and Langevin samplers as special cases. Three further, largely independent directions account for much of the remaining literature: modifying the metric, as in Riemannian HMC; modifying the Hamiltonian used by the acceptance test, as in modified or shadow HMC; and modifying the numerical integrator, as in multi-stage splitting schemes. We review these variants alongside the procedures used to tune and diagnose them, with particular attention to the effective sample size, whose standard estimators assume a reversibility that generalized and modified samplers lack, and ignore the importance weights that modified Hamiltonians introduce. We close with the open problems these methods face for hierarchical models, taking joint longitudinal and time-to-event models as the motivating example.

arXiv:2611.15607v1cs.LG

FADO: Learning Causal Explanations with Prior-Fitted Networks

In regression and classification tasks with tabular datasets, most of the literature on explanation in ML (e.g. SHAP, permutation importance) has focused on predictive attribution: does feature $X_i$ help predict outcome $Y$? At its heart, predictive attribution captures correlation, not causation. We instead ask a causal question: would manipulating feature $X_i$ change outcome $Y$? We define a natural quantity to answer this question, the feature intervention effect: $\Delta_i = E[Y\mid do(X_i=+1)]-E[Y\mid do(X_i=-1)]$. In general, estimating this quantity from observational data alone would entail strong causal assumptions. We propose FADO (Feature-wise Amortized Do-Operators), an amortized inference method using prior-fitted networks trained on synthetic tabular datasets generated by structural causal models. During training, the synthetic data-generating process is known, so we can compute ground-truth intervention effects $\Delta_i$, and train a model to predict them at test time from a purely observational tabular dataset. As we demonstrate through extensive experiments, our new causal explainability method is easy to train and provides better explanations under a range of settings for which standard predictive attribution fails, including proxy variables, confounders, colliders, and placebos. We evaluate it on real, synthetic and semi-synthetic datasets, with and without hidden confounders, and compare it to existing predictive attribution methods and state-of-the-art causal inference methods.

arXiv:2611.15456v1stat.ML

Bet Imaginatively, not Historically in Independent-Data Sequential Testing

Sequential testing avoids some of the many controversies and drawbacks related to traditional batch testing with p-values. In particular, the testing by betting paradigm has developed rapidly in recent years; it frames the level of evidence against the null as the wealth of a player in a betting game. Naturally, how much a player should bet is a key consideration, and should reflect their confidence in what they know about the next observed data point. Much current work uses standard betting strategies developed for adversarial problems. In the stochastic setting, where the observed data is independent and stationary, these approaches may be suboptimal. We propose a simple betting strategy, follow the leave-one-out e-power (FLOE), that exploits the i.i.d. nature of a data stream to estimate the e-power of our current test on the underlying distribution. We average not over the historical evidence collected so far, as generic online learning-based methods do, but instead over the equally-likely possible data sequences we might have seen based on shuffling the data. With an additional principled regularization technique, this simple drop-in method yields substantial improvements in testing performance on a variety of kernel two-sample and independence testing problems.

arXiv:2611.15300v1cs.LG

Sharp Covering-Number Bounds for Feedforward ReLU Networks via Parameter Symmetries

In neural networks, distinct parameter configurations often realize the same input--output mapping, indicating that nominal parameter counts can overstate functional complexity. In this paper, we characterize the parameter-space symmetries of ReLU and Leaky ReLU networks across fully connected, convolutional, and residual architectures. We prove that every quasianalytic parameter map preserving the network function is a composition of positive channel scaling and channel permutation. By quotienting out both symmetries, we derive uniform covering-number upper bounds in which positive scaling reduces the complexity exponent from the total parameter count $S$ to $S-H$ (with $H$ denoting the number of scalable channels) and permutation symmetry divides the bound by the factorial count $G$ of channel permutations. We establish matching lower bounds showing that the exponent $S-H$ is sharp in general and that the divisor $G$ is attained for one-hidden-layer and generically identifiable networks. We further extend these results to general continuous piecewise-linear activated networks. Finally, we apply our covering-number bounds to derive statistical risk guarantees for empirical risk minimization and capacity control beyond i.i.d.\ settings, and show how the underlying symmetries help explain weight rebalancing in function approximation and flat directions in empirical risk optimization.

arXiv:2611.15270v1stat.ML

Communication-Efficient Federated Multi-View Clustering via View-Aware Quantization

Federated multi-view clustering (FedMVC) has recently attracted increasing attention for its ability to exploit complementary clustering structures across multiple clients while preserving data privacy. However, existing FedMVC methods often suffer from high communication overhead due to the repeated transmission of high-dimensional representations, which limits their scalability in practical federated systems. Moreover, different views may contribute unequally to the clustering performance, calling for adaptive communication strategies. In this paper, we propose a view-aware quantization algorithm for communication-efficient FedMVC. By dynamically assigning quantization bits based on iteratively updated view importance scores, the proposed algorithm effectively balances clustering performance and communication efficiency under a global bit budget constraint. To validate its generality, we integrate our approach into four state-of-the-art FedMVC methods. Extensive experiments on multiple benchmark datasets demonstrate that view-aware quantization achieves up to 6.59x communication reduction while maintaining and in some cases surpassing baseline clustering performance, confirming the effectiveness and scalability of the proposed approach.

arXiv:2611.15232v1cs.LG

Winter wheat improves soil carbon sequestration with reduced gains in warmer climates

Sustainable agricultural intensification requires a precise understanding of how land use decisions influence soil organic carbon (SOC) sequestration under varying environmental conditions. We apply causal machine learning to evaluate the heterogeneous impact of winter wheat-based rotations on SOC at the field-level across Lithuania. Using a nationwide dataset collected over five years (2018-2022), including crop types, management practices, soil properties, and environmental conditions, we estimate how the effects of winter wheat-based rotation vary across space and climate. Our results indicate that winter wheat generally enhances SOC levels, but its effectiveness is modulated by climate. Cooler regions experience higher SOC gains, whereas warmer regions show diminished or even negative effects. Future climate projections suggest that the potential for winter wheat carbon sequestration may decline under high-emission scenarios. These findings highlight the importance of localized agricultural strategies that account for climate variability to maximize SOC sequestration and ensure long-term soil sustainability.

arXiv:2611.14900v1stat.ML

Nested Ensemble Learning with Topological Data Analysis for Graph Classification and Regression

We propose a nested ensemble learning framework that utilizes Topological Data Analysis (TDA) to extract and integrate topological features from graph data, with the goal of improving performance on classification and regression tasks. Our approach computes persistence diagrams (PDs) using lower-star filtrations induced by three filter functions: closeness, betweenness, and degree2 centrality. To overcome the limitation of relying on a single filter, these PDs are integrated through a data-driven, three-level architecture. At Level-0, diverse base models are independently trained on the topological features extracted for each filter function. At Level-1, a meta-learner combines the predictions of these base models for each filter to form filter-specific ensembles. Finally, at Level-2, a meta-learner integrates the outputs of these filter-specific ensembles to produce the final prediction. We evaluate our method on both simulated and real-world graph datasets. Experimental results demonstrate that our framework consistently outperforms base models and standard stacking methods, achieving higher classification accuracy and lower regression error. It also surpasses existing state-of-the-art approaches, ranking among the top three models across all benchmarks.

arXiv:2611.14892v1cs.CG

Almost Optimal Lower Bounds under Almost ETH for Approximating Euclidean k-means

In the Euclidean $k$-means problem, we are given a set of $n$ points in $\mathbb{R}^d$, and the goal is to find $k$ representative points in space so as to minimize the sum of the squared distances from each point to its closest representative. Algorithmically, de la Vega, Karpinski, Kenyon, and Rabani [STOC'03] and Kumar, Sabharwal, and Sen [JACM'10] obtain a $(1+\varepsilon)$-approximation for high-dimensional Euclidean $k$-means in time $2^{(k/\varepsilon)^{O(1)}} \cdot dn^{O(1)}$, which was subsequently improved to $2^{\tilde{O}(k/\varepsilon)} + O(ndk)$ by Feldman, Monemizadeh, and Sohler [SoCG'07]. On the lower-bound side, a recent work of Cohen-Addad, Karthik, Saulpic, and Schwiegelshohn [SoCG'26] proposed the Exponential Time for Expanders Hypothesis (XXH), which asserts exponential-time hardness for $(1+\varepsilon)$-approximating Vertex Cover on vertex expanders, and used it to prove nearly matching running time lower bound whenever $k=\tilde{\omega}(1/\varepsilon)$. We observe that XXH does not hold in the parameter regime needed for the above result, and show how to repair their reduction under a more standard assumption. More precisely, we show that assuming the Almost Exponential Time Hypothesis for 3-SAT (Almost ETH), for every constant $\beta\in(0,1)$, Euclidean $k$-means in $\mathbb{R}^d$ has no $(1+\varepsilon)$-approximation algorithm running in time $2^{(k/\varepsilon)^{1-\beta}}\text{poly}(n,d)$, whenever $k=\tilde{\omega}(1/\varepsilon)$. By modifying the construction, we also obtain a similar result for approximating the Euclidean $k$-median problem.

arXiv:2611.14887v1cs.DS

Characterizing Local Breadth in Language Generation

Large language models sharpen the validity--breadth trade-off: a generator should avoid hallucinations without collapsing onto a narrow set of valid outputs. We study generation from positive examples of an unknown target in the Kleinberg--Mullainathan model, with arbitrary, possibly adversarial order and no distributional or computational assumptions. Breadth is measured by lower Banach density, which rules out arbitrarily large sparse holes, unlike prefix-based lower asymptotic density. 1. Exact optima in one dimension: For each target $K$ in a countable family, we characterize its optimal lower Banach density $b(K)$ via equivalent combinatorial and topological formulations. One eventually valid generator attains all $b(K)$ simultaneously. Every $\alpha\in[0,1/2]$ occurs as $b(K)$ in some countable family; finite Cantor--Bendixson point rank guarantees $b(K)=1/2$, even without a uniform rank bound. 2. Unavoidable trade-offs in higher dimensions: For $d\ge2$, there need not be a single best generator: optimizing density for one target may worsen the guarantee for another. An explicit family quantifies this intrinsically multi-objective trade-off. 3. Complete characterization of density profiles: We characterize the full region of simultaneously achievable guarantees, with equivalent combinatorial and topological formulations, after excluding geometrically degenerate rectangles. Every prescribed product $\prod_i[0,\alpha_i]$, $\alpha_i\in[0,1/2]$ can occur as an exact achievable region for some language collection. 4. Near-half guarantees and the sharp endpoint: Under a near-logarithmic nondegeneracy condition, one generator achieves filtered density at least $1/2-\varepsilon$ at all finite-point-rank targets, for every $\varepsilon>0$, improving the previous polylogarithmic threshold. The exact $1/2$ endpoint can fail even for a rank-zero singleton family.

arXiv:2611.14870v1stat.ML

Synthetic Data Augmentation for Fairness under Data Imbalance: Theory and Adaptive Allocation

Data imbalance across sensitive groups and class labels is common in classification problems and can contribute to substantial fairness disparities. We study how such imbalance affects Equal Opportunity and how synthetic data augmentation can mitigate the resulting unfairness. We first identify two mechanisms, prevalence imbalance and group-size imbalance, through which changes in the training distribution shift the learned decision boundary and induce disparities in group true positive rates. Motivated by this analysis, we study exact full-balance augmentation, which equalizes all group-label cells. We establish conditions under which full balancing strictly improves Equal Opportunity together with a prediction-error comparison under the oracle balanced distribution, and show that the Equal Opportunity gap of the synthetic-data classifier converges to that of the oracle balanced classifier as the sample size grows and the generator error vanishes. The finite-sample convergence rate agrees, up to logarithmic factors and generator-approximation error, with the information-theoretic lower-bound rate under the corresponding regularity regimes. Since exact balancing need not provide the best fairness-accuracy trade-off in practice, we further propose Accuracy-Constrained Adaptive Synthetic Augmentation (ACASA), which learns both where and how much to augment by minimizing estimated fairness disparity subject to a constraint on predictive accuracy. We provide a finite-sample guarantee comparing the selected augmentation allocation with the population-optimal accuracy-feasible allocation. Simulation studies and real data application support the theoretical findings and demonstrate that synthetic augmentation can substantially reduce fairness disparities while achieving favorable empirical fairness-accuracy trade-offs.

arXiv:2611.14811v1stat.ML

Improved last-iterate bounds for SGD with and without replacement

In this work, we study last-iterate convergence bounds for stochastic gradient descent (SGD) on convex and $\beta$-smooth loss functions and improve the bound from $\cO((\log T)^3/T)$ to $\cO((\log T)^3/(T\log\log T))$ in the interpolation regime. The improvement comes from applying a horizon-dependent multiscale estimate of gradients to the comparison of trajectories. The bound applies to both sampling with replacement and sampling without replacement within one pass.

arXiv:2611.14749v1stat.ML

Don't Match the Noise: Distribution Matching under Unknown Measurement Error with an Audit Sample

Real-world machine learning systems often operate on imperfect measurements: sensors are noisy, annotations are produced by non-experts, and scientific observations are often indirect. In these settings, classic distribution comparison tools such as maximum mean discrepancy (MMD) can be systematically biased, matching artifacts of the measurement process rather than the underlying mechanisms of interest. We study a practical regime in which noisy observations are abundant, but only a small audited subset contains ground-truth measurements. We propose measurement-error-corrected MMD (MEC-MMD), an audit-based estimator that debiases kernel evaluations without requiring a noise model. MEC-MMD is exactly unbiased for the oracle MMD under simple random auditing, admits an explicit variance decomposition, and is consistent for the population MMD, achieving the standard $O_p(N^{-1/2})$ rate when the audit size scales with the dataset. We demonstrate the utility of distributional matching with MEC-MMD loss for both parameter estimation and unsupervised applications, showing that MEC-MMD consistently outperforms MMD and alternative methods. These results provide a principled foundation for robust distribution comparison, simulation-based inference, and learning in noisy real-world machine learning pipelines.

arXiv:2611.14717v1stat.ML

Robbing Peter to Pay Paul: Cross-Attribute Externalities of Fairness Interventions

Fairness interventions often targets disparities associated with a single sensitive attribute in binary classification tasks. However, when multiple sensitive attributes are associated with a data subject, improving fairness with respect to one attribute does not necessarily improve it across all attributes: it may exacerbate disparities associated with other sensitive attributes, including those that are unavailable to the practitioner. We refer to these unintended effects as cross-attribute externalities of fairness. We study how fairness with respect to one sensitive attribute varies when an initially unfair classification or regression model is corrected with respect to another attribute. We characterize these externalities using Wasserstein barycentric disparities and develop a distributional framework that captures changes in both marginal and joint fairness measures. This highlights a limitation of conventional fairness evaluation, in which the success of an intervention is assessed mainly for the attribute it targets. In a jointly Gaussian model, we derive explicit trajectories for marginal and joint disparities and identify conditions under which an intervention eliminates disparities with respect to its target attribute while increasing disparities associated with another attribute. We also derive sharp partial-identification bounds on the resulting externalities when a sensitive attribute is unavailable and extend our results to categorical sensitive attributes. We illustrate these externalities using classification and regression real-data: in the classification experiments, interventions increase unfairness with respect to the hidden attribute in the majority of scenarios across more than half of the datasets considered; in the regression experiments on the ACSIncome dataset, correcting for race reduces sex disparities, whereas subsequently correcting for age increases them.

arXiv:2611.14704v1cs.LG

Code as Explanations: Executable and Verifiable LLM Explanations

As large language models (LLMs) increasingly make consequential decisions and solve problems alongside humans, we must understand how models arrive at their conclusions. Understanding how an LLM behaves requires observing it across multiple inputs, yet free-text explanations such as chain-of-thought (CoT) are generated per instance and cannot be easily verified on new inputs. We propose {code explanations to address the mismatch in how LLM explanations are generated versus how they are evaluated: a program explains LLM behavior across a dataset of inputs, and its execution enables automatic verification. We define Consistency metric that measures the fraction of inputs on which the program reproduces the LLM's outputs, and use it to evaluate and select candidate programs. Across three open-source code-capable LLMs and eight tasks, we find that LLMs can produce highly consistent code explanations in several settings. Code explanations are also 6$\times$ to 100$\times$ shorter than CoT, and better align with model behavior in 9 out of 24 settings. However, consistency is significantly lower on tasks with more complex linguistic components, where generated programs reproduce model outputs no more frequently than an uninformative explanation baseline, and these programs often rely on longer hard-coded patterns. In experiments with Claude Opus 5.5, providing the model with consistency feedback and code execution improves explanation consistency across tasks, but consistency remains significantly lower on more complex tasks. Our findings establish executable code as a novel, verifiable, and compact approach to explaining LLM behavior and highlight verification feedback as a promising direction for improving explanation consistency. The code for this paper is at https://github.com/rajesh-lab/code-as-explanations.

arXiv:2611.14656v1cs.LG

Thompson Sampling using Prior-fitted Diffusion Transformers

Thompson sampling is natural randomized strategy for optimizing unknown functions in a black-box manner. In continuous global optimization, Thompson sampling has largely been restricted to Gaussian process priors, as these are essentially the only class for which computing the posterior distribution has been tractable in practice. In this work, we introduce \emph{prior-fitted diffusion transformers}, which are diffusion models that perform Thompson sampling in-context at inference time, after being pre-trained using the prior-fitted network paradigm. To apply these models, we develop a set of practical non-Gaussian priors for general black-box functions---including those that have characteristics that cannot be achieved under Gaussianity, such as priors over unimodal functions. We also quantify the tradeoff between model size and the number of pre-training iterations, and show that these models follow scaling laws akin to those observed in transformers defined over other modalities. Across a set of global optimization benchmarks, we show that prior-fitted diffusion transformers with non-Gaussian priors can achieve improved performance compared to traditional pipelines built on Gaussian processes.

arXiv:2611.14603v1stat.ML

An Eyring-Kramers Formula for Singular Wells

Understanding the persistence of learned behaviors in AI safety motivates an investigation of the dynamical counterpart to the equilibrium geometry studied in singular learning theory. We analyze this persistence through transitions between singular wells. We establish an Eyring-Kramers formula for a low temperature Langevin diffusion between two compact connected zero components of a real-analytic potential. The wells may be singular and nonisolated, and the unique minimizing gate is a nondegenerate index-one saddle, with gates understood as inclusion-minimal sets meeting every minimizing path. A boundary-gate construction shows that uniqueness of this inclusion-minimal gate is equivalent to every minimizing path passing through the saddle and one such path meeting the communication level only there. Compact semianalytic topology provides a separating function away from a fixed saddle neighborhood; analytic Morse coordinates give matching capacity bounds by first sending the temperature to zero and then shrinking that neighborhood. An anchored energy estimate and a small-ball potential-theoretic identity give uniformity for deterministic starting points. The leading constant is expressed in terms of the sublevel-volume coefficient, a Gamma factor, and the saddle Hessian.

arXiv:2611.14586v1stat.ML

Finite-Cadence Refresh Equilibria for Streaming Prior-Anchored Ridge: A Closed-Form Forgetting Window and Spectral Correction

Recursive estimators on drifting streams must forget stale data fast enough to track the target, yet slowly enough to average out noise. In prior-anchored ridge regression, this trade-off admits a closed-form spectral decomposition in which each eigen-direction interpolates between the data solution and the prior with pull weight $s_i = \rho / (\mu_i + \rho)$. When the anchor is refreshed every $k$ steps, we show that the moving-frame offset converges to a finite-cadence equilibrium that lags the pure-leaky path by an explicit $\Delta_k(s)$ while reducing estimation variance by a factor $\Psi(s, r_k)$. Minimizing the cycle-averaged risk gives $\omega^* = (4\nu^2 \Sigma / c')^{1/3}$, $c' = \sigma^2 d_{\mathrm{eff}}(\rho)$, and the plug-in strength $\rho^* = \sigma^2 / (n\delta^2)$. A random-matrix deterministic equivalent extends both laws to finite dimension, renormalizing the anchor to $\rho_{\mathrm{eff}} = \rho (e^{\zeta q} - 1) / (\zeta q)$; it matches simulation within 5% at fixed operating points and within 13% at the method's own, and rank-deficient populations reduce exactly to their active subspace. The resulting method, SPAR, calibrates window and strength online from residuals. On a 27-configuration grid the calibrated window opens on drift without divergence, and the grid defaults transfer to an hourly electricity stream and to BCI IV 2b. The core algebra, refresh chain, and regret bounds are formalized in Lean 4.

arXiv:2611.14540v1stat.AP

Katz Kernel Based Stratification of Breast Cancer Patients

The integration of mechanistic signaling networks into clinical research has become a cornerstone of precision oncology. By capturing the complex biochemical interactions within a cell, these networks provide a functional map of disease progression that goes beyond simple genomic mutations. Consequently, the development of specific signaling models of patients has significant relevance, allowing for a more granular representation of individual disease profiles. In this setting, there is a growing need for robust methodologies capable of comparing graph based pathway structures to identify similarities and differences across patient cohorts. Such comparisons are essential for the identification of clinically meaningful subgroups, and the discovery of novel biomarkers. Standard graph kernels, while mathematically rigorous, often remain too generic for this purpose. They frequently fail to capture features specific to gene signaling networks, resulting in suboptimal clustering performance. In this paper, we present a domain specific graph kernel designed for mechanistic signaling networks. Unlike generic approaches, our method explicitly incorporates biological constraints and gene interaction patterns into the kernel formulation, enabling a more accurate characterization of both topological structure and functional behavior. We assess its performance on breast cancer datasets, focusing on its ability to support disease subtyping. Our results suggest that our approach can effectively contribute to patient stratification.

arXiv:2611.13219v2stat.ML

MediPFN: Amortized In-Context Nuisance Learning and Inference for Causal Mediation Analysis

Causal mediation analysis decomposes a treatment effect into indirect pathways through mediators and direct pathways through other processes. Tabular foundation models perform classification and regression through in-context learning, providing estimates of conditional means and propensity scores used in causal mediation analysis. Constructing confidence intervals from these estimates requires accounting for uncertainty introduced by fitting and combining several nuisance functions. We introduce MediPFN, an amortized inference framework that adapts a pretrained tabular foundation model to nuisance prediction and combines its predictions through a cross-fitted influence-function estimator. A separate head learns quantiles of the estimator's standardized error from simulated tables with known indirect effects. Supervised fine-tuning, symmetrization, and independent calibration then produce a confidence interval. We establish coverage bounds for the resulting intervals under standard assumptions. Across experiments, MediPFN attains coverage closer to the nominal level than several comparators. It also reduces the computational cost of uncertainty quantification, running 59.1 times faster than bootstrap in matched comparisons.