Let $P$ be an irreducible reversible Markov kernel on an $m$-state space $\Omega$, and denote its right spectral gap by $\Gap(P) =1-\lambda_2(P)$. For a stationary trajectory, let $K_t$ be the occupation-count vector of the length-$n$ window beginning at time $t$. The stationary pair $(K_0,K_1)$ defines a reversible projected count kernel $\widetilde P_n$. For each fixed $m\ge2$, define \[ c_m^\star= \inf_{\substack{|\Omega|=m,\ P\ \mathrm{irreducible\ and\ reversible} n\ge2}} \frac{n\Gap(\widetilde P_n)}{\Gap(P)}. \] We prove \[ \frac1{1080m}\le c_m^\star\le q_{m-2}, \qquad q_0=\frac14,\quad q_{r+1}=q_r(1-q_r). \] We also show $q_{m-2}=(m+\log m+O(1))^{-1}$, which implies that $c_m^\star=\Theta(m^{-1})$. Thus, the reciprocal optimal comparison coefficient $C_m^{\mathrm{opt}}=(c_m^\star)^{-1}$ is of the order $m$. In the two-state case, we determine the exact values $c_2^\star=1/4$ and $C_2^{\mathrm{opt}}=4$. %the exact constants for $m\ge3$ remain open. The lower bound also holds for $n=1$ and is uniform in $P$, including sparse and periodic kernels. The proof combines short-window decorrelation with an averaged anchor-excursion decomposition, a Green-kernel hitting estimate, and a stopped Carleson--Hardy inequality. A nested rare-state construction produces finite $m$-state witnesses whose normalized Rayleigh quotients approach $q_{m-2}$ through an ordered sequence of limits. For every fixed finite irreducible reversible aperiodic kernel on at least two states, $\Gap(\widetilde P_n)=\Theta_P(n^{-1})$.
Large language model (LLM) distillation can propagate teacher bias under covariate shift when target reward feedback is unavailable. We propose Iterative Calibration and Learning (IterCaL), which alternates source-reward teacher calibration through token-level branching with student learning on target prompts. We prove polynomial convergence in expected average KL to the oracle student, the reference-regularized target-reward maximizer within a potentially misspecified student class; a quantified source-to-target calibration discrepancy gives an explicit additive true-return term. Our separation shows that regularized direct teacher matching retains positive oracle-KL error at its population optimum for every shared reference penalty, even when the teacher is stronger than every student. On the same instance, IterCaL converges to the oracle using source rewards alone.
Binary Iterative Hard Thresholding (BIHT) is a simple, yet effective, greedy method for recovering a sparse vector from one-bit sign measurements. In its original form, BIHT performs a ``gradient-descent'' step, followed by hard thresholding. A convergence analysis of this algorithm was left open in the introductory work of Jacques, Laska, Boufounos, and Baraniuk [IEEE Trans. Inform. Theory (2013)] and has remained unresolved for over a decade, with subsequent sharp analyses studying a normalized variant instead, that additionally projects every iterate onto the unit sphere. This paper resolves that gap and characterizes when per-iterate normalization is algorithmically necessary. In the noiseless setting, we prove a universal, sample-optimal convergence theorem for the original BIHT algorithm. Specifically, with $\widetilde O(s/\epsilon)$ measurements, a deterministic, computable finite-time iterate has directional error at most $\epsilon$, simultaneously for every $s$-sparse unit vector. This matches the optimal sample dependence achieved by normalized BIHT in prior work. Under sign corruptions, we prove a sharp separation. If at most a $\tau$ fraction of signs are flipped adversarially, then BIHT, without per-iterate normalization, still reaches the robust error floor at an early iterate with a matching $\widetilde O(s/\epsilon)$ sample complexity rate as its normalized variant. This recovery, however, is not stable. We prove a scalar lower bound showing that any nontrivial corruption pattern, even one that involves only one flipped sign together with one clean sign, forces the iterates to oscillate indefinitely. Consequently, no general last-iterate convergence theorem can hold for BIHT under sign corruptions, while its normalized surrogate provably escapes this instance.
Survival analysis provides statistical methods to model the time until an event occurs. Reporting delays arise when event times are not observed at their occurrence but are only revealed upon reporting. This issue is particularly critical for timely risk evaluation when the observation window is short due to administrative censoring. In this study, we incorporate right-censored reporting delays by jointly modeling parametric hazards for the event and reporting processes. We then construct a consistent estimator for the model parameters and develop a Monte Carlo expectation-maximization algorithm to compute it. To address the challenges posed by administrative censoring, we leverage these findings and propose a transfer-learning procedure. Experimental results demonstrate that our method improves the accuracy of timely risk evaluation under administrative censoring.
Flow Matching has recently gained attention in generative modeling as a simple and flexible alternative to diffusion models. While existing statistical guarantees adapt tools from the analysis of diffusion models, we take a different perspective by connecting Flow Matching to kernel density estimation. We first verify that the kernel density estimator matches the optimal rate of convergence in Wasserstein distance up to logarithmic factors, improving existing bounds for the Gaussian kernel. Based on this result, we prove that for sufficiently large networks, Flow Matching achieves the optimal rate up to logarithmic factors. If the target distribution lies on a lower-dimensional manifold, we show that the kernel density estimator profits from the smaller intrinsic dimension on a small tube around the manifold. The faster rate also applies to Flow Matching, providing a theoretical foundation for its empirical success in high-dimensional settings.
Synthetic data are widely used for training modern AI systems. However, their statistical understanding remains limited. It is well recognized in statistics that repeated self-training on synthetic data can amplify errors inherited from pre-trained models, leading to model collapse. Verification plays a crucial role in selecting high-quality synthetic data to help prevent model collapse and improve model performance. We study verified self-training in a linear regression framework, where a pre-trained estimator generates synthetic data with injected generation noise and a verifier selects samples according to a verification threshold. Post-training on the accepted samples updates either all coordinates (full fine-tuning) or only an active subset (partial fine-tuning). Our analysis reveals three potentially counterintuitive effects. We establish that tighter verification does not always lead to lower prediction risk. We also find that a suitable level of injected generation noise can reduce prediction risk, while excessive noise can increase it. Moreover, full and partial fine-tuning can respond differently to the same verification rule, since pre-training error in the frozen coordinates affects both sample selection and post-training risk. We analyze these effects by deriving finite sample risk bounds for four separate regimes determined by the relative scales of the verification threshold, generation noise, and pre-training error. The bounds precisely characterize how verification affects prediction risk across these regimes. In particular, we quantify how the generation noise, the effective accepted sample size, and the covariance of the accepted covariates affect prediction risk. The analysis extends to iterative verified full fine-tuning, where sufficiently tight verification reduces the initial error contribution at a geometric rate.
We study high-dimensional nonparametric regression under multisource covariate shift, where $\mathrm J$ sources of training data share a common regression function but differ in their covariate distributions, sample sizes, and noise variances. Our goal is to combine these sources to estimate the regression function accurately under a target covariate distribution. We propose an oracle weighted neural estimator and establish its minimax optimality up to logarithmic factors through matching upper and lower bounds. The resulting rate is $\mathsf n_{\rm eff}^{-2p/(2p+K)}$, where $p$ and $K$ denote the effective smoothness and dimension of the regression function, and the effective sample size is \begin{align} \mathsf n_{\rm eff} =\sum_{j=1}^{\mathrm J} \frac{n_j}{(1+σ_{ρ,j})σ_{ε,j}^{2}}. \nonumber \end{align} Here, $n_j$ is the sample size of source $j$, $σ_{ε,j}^{2}$ is its noise variance, and $σ_{ρ,j}$ controls the deviation between the covariate distribution of source $j$ and the target covariate distribution. We also develop an adaptive procedure that requires no knowledge of source noise variances or covariate-shift parameters. Under suitable conditions, this procedure attains the oracle minimax rate up to logarithmic factors.
We establish structural properties of the nonparametric maximum likelihood estimator (NPMLE) for $d$-dimensional Gaussian mixtures whose means lie in a hypercube. We show that, with high probability, the NPMLE based on $n$ independent observations self-regularizes by fitting a mixture model with $O((\frac{\log n}{\log\log n})^d)$ components, achieving the optimal model complexity without explicit model selection. Furthermore, the NPMLE exhibits a structural adaptation property: For data drawn from a finite mixture, the number of fitted components is bounded in probability as $n\to\infty$. Our proof applies a multivariate Jensen-type bound to control the typical number of critical points of the KKT function associated with the first-order optimality conditions. The key step is to lower bound its gradient norm uniformly on complex spheres. We establish this lower bound by decomposing the KKT function into an empirical fluctuation term and a relative density estimation error term, combining small-ball probability estimates for the former with polynomial approximation and Hellinger error bounds for the latter.
Estimating conditional parameters, such as heterogeneous treatment effects, often requires rich covariate adjustment, even when the parameters are reported along only a few dimensions. Fitting nuisance functions on the full sample makes all observations available for adjustment, but creates dependence between fitted scores and the observations at which they are evaluated, which is typically avoided by cross-fitting. We develop pointwise inference with full-sample nuisance predictions for exactly identified conditional moment equations, recovering conditional average treatment effects as a special case. We localize the estimating equations around the reporting point using distributional nearest neighbors (DNN), a U-statistic-based averaging procedure. Under sufficient conditions on the local design, orthogonality, prediction accuracy and quantitative deletion stability, the estimator is asymptotically normal, and a frozen-score jackknife consistently estimates its limiting covariance with nuisance predictions held fixed. In simulations, full-sample and cross-fitted inference perform similarly, with coverage shortfalls from nuisance approximation error, spatial bias and heavy studentized tails. An application to 401(k) eligibility using Survey of Income and Program Participation data finds similar estimates across nuisance prediction methods, with wide normal-reference intervals.
Kernel methods face two recurrent bottlenecks: the cubic cost of inverting an $n\times n$ kernel matrix, as in kernel ridge regression, and the evaluation of kernel integrals, as required in kernel quadrature. Existing low-rank kernel and Gaussian process approximations primarily address the former bottleneck, but do not control the downstream quadrature error when both the kernel matrix and kernel mean are approximated, particularly when the integrand lies outside the kernel reproducing kernel Hilbert space. We address both challenges through an operator-adapted finite-feature approximation. Specifically, we construct a finite-dimensional working kernel from the eigenvalues and eigenfunctions of a second-order differential operator. We establish convergence guarantees for the resulting finite-feature estimators in kernel quadrature, with extension to other related kernel methods. Our results show that, with $m \lesssim n$ basis functions, the truncated estimator attains the corresponding benchmark statistical rates. The theory covers fixed/random designs, noisy/noiseless observations, oversmoothing/undersmoothing regimes, and Mat\'ern/Gaussian-type weights. We provide explicit error bounds that quantify the roles of the spectral cutoff, regularization parameter, lengthscale, sampling geometry, and operator boundary condition. We verify the convergence results in numerical experiments.