How many observations are needed to estimate a common rotation in two-dimensional overcomplete independent component analysis? We study equally spaced mixing columns with known source count, common source law, and column norm. For a fixed non-Gaussian fourth-order Gram-Charlier source law, the rotation is identifiable modulo the column spacing. Yet the sample size for fixed relative angular accuracy grows exponentially with the source count. We characterize this sample size, up to absolute constants, by the reciprocal of the squared Hellinger distance between two rotations separated by half the column spacing. An explicit radially weighted phase estimator attains this scale uniformly over the rotation family, using the observation radius to prevent cancellation in the mean of an angular harmonic. We also determine the exact exponential decay rate of single-observation total variation for the same pair. This gives bounds on the sample size exponent; its exact value remains open. Finally, a noisy binary-source example shows how an exact zero line of the observed characteristic function can have an exponentially small normal derivative.
Wireless foundation models learn reusable representations across tasks, configurations, and deployment environments. Their pretraining requires environmental breadth, yet wireless datasets often sample few sites densely. We introduce Channels4World, a scene-centered dataset of 100,000 distinct outdoor RF scenes and a procedural method for generating them. Each scene links a generated environment and RF material assignments to 32 base-station (BS) positions, 16,384 UE positions, and aligned variable-length multipath and ordered interaction records at 3.5 and 7~GHz. The corpus comprises 52.43 billion physical BS-UE pairs, 104.86 billion frequency-specific evaluations, and a 29.14-TB packaged payload. Scene identity aligns geometry, placements, channels, and interactions across frequencies, allowing environmental breadth and within-scene observation density to vary independently. The public release offers predefined subsets at multiple scales and separate access to corpus components. A deterministic 90K/10K scene-level train-test split keeps both frequencies and all scene components together. We characterize morphology, RF materials, and transceiver placement across three generation profiles. Controlled propagation and path-retention analyses show that diffraction materially affects retained-link availability and multipath structure, while the retention policy reduces storage substantially with little loss in represented aggregate power. Under matched training exposure, models trained on broader scene collections perform better on held-out scenes, although an equal-episode control shows that scene count alone cannot explain the gain. By making environmental breadth explicit and controllable, Channels4World supports the development and evaluation of wireless foundation models for generalization across deployment environments.
Resource-constrained IoT devices cache learned models for real-time decisions, but frequent updates incur substantial bandwidth, energy, and service overhead. Because decision-relevant model differences need not grow with elapsed time, we define Age of Intelligence (AoInt) as a model-version-relative, client-specific, and action utility distortion, and formulate synchronization as a tradeoff between stale-decision regret and refresh cost. Furthermore, we develop Kullback-Leibler-aware AoInt Synchronization (KL-AoIntSync), which accumulates output-distribution changes using client-specific unlabeled probes, whereas Parameter-aware AoInt Synchronization (Param-AoIntSync) uses a server-broadcast scalar parameter change. Then, we establish pathwise guarantees with fixed client distributions, action sets, and utilities. The numerical results show that calibrated Param-AoIntSync reduces joint synchronization cost by 6.15% relative to an Instant-Parameters policy while performing comparably to KL-AoIntSync.
How many environments and samples does it take to separate $d$ Gaussian sources whose variances change across environments? Each source has a signature, its vector of variances. The Riesz $2$-energy $E_j$ of source $j$ sums the inverse Fisher informations for rotating it towards other sources. With known whitening, we prove a worst-case squared-angle risk lower bound of order $E_j/(n+E_j)$ for orthogonal estimators at every $n$ and $d$. A quadratic pencil given the signatures attains $E_j/n$ up to a constant above a separation-dependent sample threshold, even with estimated whitening. For well-spread signatures at fixed $K$, the energy grows as $d^2$, $d\log d$ and $d$ for $K=2$, $3$ and $K\ge4$, respectively. A design with $K\asymp\log d$ and pairwise Fisher information bounded below by a constant makes $d\log d$ samples sufficient. The eigenvectors of a single fixed linear combination have an average first-order error coefficient of at least order $d^2$. Simulations up to $d=1000$ support the first-order predictions, and on task fMRI the energy helps rank split-half instability.
For a stochastic program $\min_{z\in Z} c^T z + h(z)$ with a random cost vector $c$, regret, or the expected value of perfect information, is usually computed by re-solving the problem scenario by scenario. We show that regret is determined by the covariance between the cost vector and the optimal decision $\pi^s(c)$. The identity is exact and holds without regularity assumptions once the covariance is integrated along the path from the mean to each scenario. With $c_s=E[c]+s(c-E[c])$, $Reg=-\int_0^1 s^{-1} Cov(c_s,\pi^s(c_s))\,ds$. Three consequences follow. First, the plain covariance is always an upper bound: $0\le Reg\le-Cov(c,\pi^s(c))$. Second, when the optimal decision rule is affine, as in unconstrained and equality-constrained (e.g., budget-constrained Markowitz) quadratic programs, regret equals exactly one half of the covariance. For purely linear objectives, an affine rule forces zero regret. Third, the ratio of regret to covariance lies in $[0,1]$ and is governed by the shape of the covariance profile along the path. The noise level at which this profile first becomes positive is a decision-stability margin that tightens the bound. The identity also holds scenario by scenario for any predicted cost, so it applies to linear and combinatorial programs with discontinuous decision rules. We give dimension-free concentration bounds and a central limit theorem for the plug-in estimator. Across 7 problem classes, 8 cost distributions, and the shortest-path benchmark of Elmachtoub and Grigas (2022) with $10^6$ draws per configuration, the identities hold to within quadrature error, and the covariance bound holds in every scenario.
Steinke and Zakynthinou(2020) introduces the Conditional Mutual Information (CMI) framework of analyzing the information complexity of learning algorithms based on algorithm-dependent information-theoretic quantities. We study one of these quantities, the evaluated Conditional Mutual Information (eCMI). It has been an interesting question whether the optimal PAC guarantee for VC classes can be recovered from the algorithm-dependent analyses via CMI. And we show that it is possible to recover this guarantee by constructing a learning algorithm whose eCMI is of order O(d) in the realizable case, where d is the VC-dimension of the concept class. Specially, our algorithm is a randomized Majority-of-5 base learners with optimal in-expectation generalization guarantee.
A fundamental question in physics is: When does classical behavior emerge from quantum systems? Bosonic Gaussian states provide a natural setting to explore this quantum-classical boundary, as they capture both the classical field behavior and the intrinsic quantum nature of light. Here, we address this problem from a learning-theoretic perspective by asking: When are bosonic Gaussian states classical to learn? That is, under what conditions (if any) can an n-mode bosonic Gaussian state be learned with as few samples, and with operations as simple, as are needed to learn a classical 2n-variate Gaussian distribution? We establish a smooth crossover in learnability governed by the state's thermal fluctuations: - Cold Gaussian states are non-classical to learn: When the covariance matrix satisfies $\Sigma\le(\frac12+O(\frac1n))I$, i.e. close to the vacuum covariance, tomography under single-copy (i.e., non-entangled) measurements fundamentally requires $\Omega(n^3)$ copies, strictly exceeding the sample complexity $\Theta(n^2)$ of learning classical Gaussian distributions. We show that this hardness persists even when few-copy entangled measurements are allowed. - Warm Gaussian states are classical to learn: When thermal fluctuations exceed the vacuum noise, parameterized by $\Sigma\ge(\frac12+\nu)I$ for any parameter $\nu>0$, we prove that single-copy tomography requires $N=\Theta\left(n^2\min(n,1+\nu^{-1})\right)$ copies. This bound is tight and is achieved by simple, non-adaptive, unentangled heterodyne measurements. Crucially, for $\nu=\Omega(1)$, the sample complexity drops to $\Theta(n^2)$, matching the classical case. Our results tightly characterize a quantum-to-classical crossover in the learnability of bosonic Gaussian states, reveal a novel connection between fundamental physics and statistical learning theory, and have implications for real-world sensing experiments.
Spectral contrastive learning keeps the $k$ directions along which two views of the data depend most strongly, while downstream tasks may need other directions. Existing guarantees for spectral features assume that the distribution of tasks has no preferred direction. We show that this isotropy condition is the wrong criterion. With linear probing, the task distribution affects the average probing gain only through the matrix $B\Lambda B^\top$, where $B$ records the dependence between views and $\Lambda$ is the task covariance: optimal features span its top-$k$ eigenspace, whereas spectral features span that of $BB^\top$. Transfer therefore depends on alignment, and strongly uneven task preferences cost nothing when they favour the spectral directions. Otherwise the regret of spectral features is at most linear in the anisotropy of $\Lambda$. No representation chosen without task information improves this rate when the leading spectrum is flat, and a spectral gap makes it quadratic. With labelled tasks, the same matrix gives a correction: Fisher discriminant directions for a frozen encoder, or a one-term change to the spectral contrastive loss. We bound the regret of the correction by the error in estimating $B\Lambda B^\top$, and compare this error with the misalignment of spectral features to decide when to correct. The analysis extends to finite samples, continuous variables and noisy views. Correction reduces synthetic held-out regret from $0.858$ to $0.003$ with $2000$ labelled tasks, and the decision rule gives the lowest regret among the tested rules at every bank size. On CIFAR-100, spectral features of frozen encoders keep most of the task signal on unseen classes, and where a bank of other classes would lower accuracy, the rule keeps them in every case.
We study the query complexity of sampling from high-dimensional Gaussian distributions using gradient information. In the standard oracle model, exact gradients expose only matrix-vector products with the precision matrix, leading to polynomial approximation barriers and a characteristic $\sqrt{\kappa}$ dependence on the known spectral range. We show that this barrier disappears when the sampler is allowed to query smoothed scores, namely gradients of the logarithms of the Gaussian-convolved densities. For a centered Gaussian target with precision matrix $\Lambda$, a smoothed-score query at noise level $\tau$ gives access to the resolvent $(\Lambda+\tau^{-1}I)^{-1}$. Combining geometrically spaced noise levels with sinc-quadrature rational approximation, we obtain a sampler with $q=O((\log\kappa+\log(e\sqrt d/\delta_{\rm TV}))\log(e\sqrt d/\delta_{\rm TV}))$ smoothed-score queries for total variation error $\delta_{\rm TV}$, improving the condition-number dependence from $\sqrt{\kappa}$ to logarithmic. We also study finite-bit gradient oracles. Using coordinatewise quantization of the transformed smoothed-score answers and a final dithering step, we obtain a sampling scheme whose total communicated gradient information is polylogarithmic in $\kappa$; in particular, for fixed dimension and accuracy, the bit complexity is $O(\log^2\kappa)$. To complement these upper bounds, we introduce a channel-synthesis, or reverse-Shannon, converse technique for sampling lower bounds. This converts total-variation simulation guarantees into communication requirements and yields an $\Omega(d\log\kappa)$ bit lower bound for $d\ge2$ and fixed total variation error below one, as $\kappa\to\infty$. Together, these results identify smoothed scores as a provably more informative oracle for sampling and give nearly matching upper and lower bounds for its finite-bit complexity.
\emph{Kullback-Leibler} (KL) regularization is ubiquitous in reinforcement learning algorithms in the form of \emph{reverse} or \emph{forward} KL. Recent studies have demonstrated $\epsilon^{-1}$-type fast rates for decision making under reverse KL regularization, in contrast to the standard $\epsilon^{-2}$-type sample complexity. However, for forward-KL-regularized objectives, existing statistical analyses are either not applicable or result in $\tilde{O}(\epsilon^{-2})$ slow rates. We take the first step towards addressing this problem via a streamlined analysis of forward-KL-regularized offline CBs. We give the first $\tilde{O}(\epsilon^{-1})$ upper bounds in tabular and general function approximation settings, both under notions of \emph{single-policy concentrability}. In particular, our convex-analytical pipeline unifies these settings by exploiting the pessimism principle in a novel way and completely bypasses the proof routines in previous works based on the mean value theorem, which might be of independent interest. Moreover, we provide rate-optimal lower bounds, manifesting the tightness of our upper bounds in terms of statistical rates. Our lower bounds also demonstrate that the forward-KL-regularized sample complexity recovers the unregularized slow rate in the low-regularization regime, similarly to the reverse-KL regularization.
Agents built on large language models (LLMs) rely on reliability techniques such as retry, majority voting, and self-consistency developed in parallel rather than within a common analytical framework. We observe that an LLM sampled at temperature $T$ is a discrete stochastic channel $p(y \mid x)$ in the sense of Shannon's coding theory, and use this identity as the entry point for such a framework. Each of these techniques is a special case of one of six classical reliability operators: diversity combining, hybrid retransmission, iterative generator-critic decoding, rateless sampling, structured redundant verification, and difficulty-adaptive routing. Within the framework we give two closed-form results, a noise-variance threshold above which uniform averaging beats quality-weighted averaging and a contractivity criterion for generator-critic refinement, and turn each into a design rule checkable before committing compute. We further introduce a cost-aware semantic-nearest-neighbor router whose single Lagrangian knob traverses the quality-cost frontier without retraining. Across six local and cloud channel configurations, no fixed model-technique-budget choice dominates. On a 300-item hard split of MMLU, GSM8K, and HumanEval, the router occupies the empirical Pareto frontier: at matched quality its normalized cost is $\approx$56% lower than the best fixed technique, and at matched cost it improves quality by $\approx$7% (26% over single-shot decoding). Judge-free grading of 6,900 outputs confirms that the judge tracks correctness at the item level and that the router's advantage over a fixed policy persists at most cost settings; with internal reasoning enabled, the gain of external refinement shrinks but remains significant for our incremental-redundancy operator. These results argue for consolidating reliability techniques into a single tunable layer informed by channel coding.
Modeling high-dimensional dependencies while keeping likelihoods tractable re- mains challenging. Classical vine copulas provide an explicit dependence structure but require repeated, costly bivariate fitting. We propose Vine Denoising Copula (VDC)1, an amortized estimator for simplified vines on continuous data. VDC trains a single bivariate denoising network on simulated copulas and reuses it across all vine edges. For each edge, the network maps an empirical histogram to a density grid, and iterative proportional fitting approximately enforces uniform grid marginals. The predicted densities and cached conditional transforms support stan- dard vine recursion, likelihood evaluation, sampling, and information estimation. This retains the classical vine factorization while replacing repeated per-edge fitting with reusable neural inference. Across synthetic and real-data benchmarks, VDC combines accurate bivariate density estimation, competitive information estimates, and faster high-dimensional vine fitting after pretraining. Performance varies across datasets; finite grid resolution and conditional prediction remain limitations. The explicit fitted structure supports edge- and tree-level information decomposition, making repeated dependence analysis more practical.
We formulate an algorithm-independent theory of learning dynamics for individual models and ensembles. We establish a conservation law, the vanishing of predictive entropy under successful learning, quantitative entropy and core-expansion bounds, and calibration and finite-sample validation results for identifying reliable predictions. These results motivate a two-generation ensemble that uses Jensen-Shannon (JS) disagreement to route uncertain inputs to specialized learners. CIFAR-10 experiments examine controlled perturbations, distribution shifts, and adversarial attacks. On a heterogeneous test environment, routing improves accuracy over probability averaging while keeping core accuracy near 95%. For adversarial defense, we consider a query-only black-box setting: the attacker can query the overall prediction function but does not know a particular internal decomposition into learners and routing rules. Our current attacks use a fixed-initial-route gradient, which is accurate for a single-step attack when the route remains unchanged, but may deviate from actual multi-step attacks, especially for large perturbations. We evaluate our two-layer defense against this approximation; whether additional adversarially trained layers improve resistance to stronger multi-step attacks remains for future study.
Can a sender that only computes hashes, with a language model only at the receiver, transmit short human text more cheaply than a classical sender trained on the same domain? We answer with one pre-registered run on 1,000 English user utterances from Taskmaster-1. The primary endpoint is the cost per message to deliver 100% of the messages (hash bits, plus a 2-byte negative acknowledgement and the raw text when the first attempt fails), paired against an in-domain PPM order-1 sender. The hash-only sender delivered 857 of 1,000 messages on the first attempt and cost 192.8 bits per message, 18.4 bits more than PPM order 1 (95% bootstrap interval 10.9 to 26.3); by the pre-registered rule it spends more and is not equivalent within one byte. As secondary results we map measured sender memory against bits. The hash-only sender (8,190 B on a host build, keyed BLAKE2b tag included, as in every method) was the cheapest point in bits from 8,190 B up to, but excluding, PPM order 1 at 24,844 B, 45 to 46 bits below smaz2 and Unishox2; from PPM order 1 upwards, in-domain senders spend fewer bits, except order 2 pruned to 16 KiB. In airtime it loses to PPM order 1 on all five links priced, to Unishox2 on four, and to raw text with a tag on two, because each failure costs an acknowledgement frame and a fallback frame. A wider receiver search recovers half of the failures and makes the cost 7.7 bits lower than PPM order 1, at about three minutes of receiver time per failure, and still loses in airtime on four of five links. We report the measurement as a delimitation, with every deviation of the run.
We study planning for finite-horizon partially observable Markov decision processes (POMDPs) with finite state, action, and observation spaces. Both exact and approximate planning are known to be computationally hard in this setting, so prior work has explored natural assumptions that make planning easier. Recent work [Golowich et al., 2022] gave quasipolynomial-time approximate planning algorithms for observable POMDPs. These are POMDPs whose observation channels satisfy an $\ell_1$-observability condition, but notably, no assumptions are required on the transition dynamics. In a similar vein, for every fixed $\rho>0$, we give quasipolynomial-time approximate planning algorithms for $\rho$-decodable POMDPs. We show that decodability is implied by both $\ell_1$- and $\ell_2$-observability and is strictly weaker than observability. Moreover, we show that the optimal decodability constant can be computed in polynomial time, whereas computing the optimal $\ell_1$-observability constant is NP-hard in the worst case. We also show that the fast filter-forgetting guarantee underlying the approach of [Golowich et al., 2022] does not extend directly from $\ell_1$-observability to constant $\ell_2$-observability. Thus, decodability is an easy-to-check, unifying assumption under which quasipolynomial-time approximate planning for POMDPs is possible.
We study sequential prediction of binary marks whose log-odds are a linear function of all past inputs. The inputs are independent uniform signs. The coefficient of the input observed $j$ steps earlier is unknown, but its absolute value is at most a known number $r_j$, and $\sum_j r_j\le B$. A learner is judged by its expected cumulative log loss in excess of the loss of the true conditional probabilities. We show that for every such sequence $r$ and every horizon $T$, the minimax value of this regret lies between $c\,\Gamma_T(r)/\max\{1,B^2\}$ and $\frac12\Gamma_T(r)$, where $\Gamma_T(r)=\sum_{j\le T}\log(1+(T-j+1)r_j^2)$ and $c$ is an absolute constant. The term for lag $j$ combines the size bound $r_j$ with the number $T-j+1$ of rounds in which that lag influences a prediction, and no monotonicity or other regularity of $r$ is needed. The upper bound is attained by a Bayesian mixture with independent Gaussian priors. For the lower bound, we apply the van Trees inequality at each prediction step and then bound the expected log-determinant of the random matrix whose rows are the lagged input vectors. We also characterize when $\frac12$ is the exact leading constant: for envelopes with a common bound $B$, possibly changing with $T$, the regret is asymptotic to $\frac12\Gamma_T(r)$ if and only if $\sum_{j\le T}\min\{1,(T-j+1)r_j^2\}=o(\Gamma_T(r))$. This gives, for example, $\log^2(A^2T)/(8\alpha)$ for $r_j=Ae^{-\alpha j}$, whereas for $r_j=Aj^{-s}$ with $s>1$ the regret is of order $T^{1/(2s)}$ and its ratio to $\Gamma_T(r)$ stays below $\frac12$. Finally, for a known horizon, an online Newton predictor with features scaled by $r_j$ attains regret of order $\Gamma_T(r)$ with polynomial computation per round.
Evaluating large language models increasingly relies on LLM-as-a-judge protocols, but such evaluations remain costly: different judges have different prices and reliabilities, and the difficulty of each prompt-response pair can vary substantially. This raises a basic allocation question: under a fixed budget, how should one distribute evaluation queries across heterogeneous judges and instances to obtain the most accurate score estimates? We formalize this question as budgeted heteroskedastic multi-judge estimation under independent, unbiased score observations. Given $K$ prompt-response pairs and $J$ judges with known costs but unknown query-judge variances, our goal is to estimate the bounded score vector while minimizing an $\ell_p$-error. Our first contribution is to analyze the inverse-variance weighted estimator (IVWE) and to derive the oracle allocation that minimizes its leading error bound. Since this allocation depends on the unknown variances, we then address the unknown-variance setting by proposing EST-IVWE, an adaptive algorithm that constructs and uses optimistic variance estimates to stabilize the empirical allocation. We prove that EST-IVWE matches the oracle IVWE rate up to lower-order terms in the budget. Our second and central theoretical contribution is a local minimax lower bound matching the leading allocation-dependent rate, establishing asymptotic instance-optimality up to logarithmic factors. An Assouad-type construction based on local, variance-preserving perturbations yields this bound both in expectation and with constant probability. A complementary Fano-type change-of-measure argument captures the logarithmic dependence on the confidence level with a different (suboptimal) allocation geometry. Finally, we numerically validate our approach on synthetic data and resampled HelpSteer2 evaluations, demonstrating improvements over na\"ive uniform allocation.
The success of large foundation models is catalyzing a new paradigm for AI-native 6G network design: wireless foundation models for physical-layer design. However, existing models often operate on channel state information (CSI) in the spatial-temporal-frequency (STF) domain, where multipath components are superimposed and structurally entangled. This hinders the learning of a universal channel representation. Their reliance on global attention also incurs prohibitive overhead. In this paper, we propose AirFM-DDA, an Air-interface Foundation Model in the Delay-Doppler-Angle (DDA) domain. AirFM-DDA reparameterizes CSI into the DDA domain to resolve multipath components along physically meaningful axes and employs window-based attention with frame-structure-aware positional encoding. Extensive experiments demonstrate transferability across scenarios, tasks, datasets, and antenna configurations. For channel prediction and estimation, AirFM-DDA generalizes zero-shot to unseen cities, achieving average normalized mean-square error (NMSE) gains of 4.9-8.5 dB over the strongest baselines. With only 10% labeled data, it achieves average gains of 12.0 percentage points in Top-1 accuracy for beam prediction and 3.4 percentage points in F1 score for line-of-sight (LoS) identification. It further transfers across simulated datasets and adapts to measured data and different antenna arrays. Compared with global attention, window-based attention reduces training and inference costs by nearly an order of magnitude.
Flow-based generative models have emerged as an efficient alternative to diffusion models. We study the sample complexity of learning a target distribution in Wasserstein distance via flow matching, which learns a velocity field transporting a base distribution to the target. Under exact empirical risk minimization over a fixed network class and standard assumptions on the architecture and data distribution, we show that any squared-loss flow-matching variant including rectified flow with bounded derivatives achieves $\widetilde{O}(\varepsilon^{-2})$ sample complexity for the statistical term with the Wasserstein guarantee taking the form $W_2 \le K(\varepsilon + C\varepsilon_{app})$, where $\varepsilon_{app}$ is the approximation error of the network class. This improves upon existing $\widetilde{O}(\varepsilon^{-4})$ guarantees and does not require the Polyak--Lojasiewicz condition in the statistical analysis. Instead, we exploit a structural property of the squared-loss objective - it induces an approximate Bernstein condition that directly ties the variance of the excess loss to the excess risk up to additive approximation-error. This condition enables a localized Rademacher complexity analysis yielding fast $O(1/n)$ rates. The flow-matching schedule enters the leading statistical prefactor through the Lipschitz constant of the pointwise loss, which we characterize explicitly for standard schedules.
Semantic communication (SemCom) using a relay node is considered, where the relay assists in conveying task-relevant information from a source to a destination, with meaning defined by classification labels. A joint end-to-end design of the source encoder, relay transformation, and destination decoders is developed to optimize both semantic inference accuracy and perceptual reconstruction quality for image data. Relay-assisted schemes are shown to outperform direct transmission, with gains that become more pronounced in low signal-to-noise ratio (SNR) regimes across different relaying strategies. Alongside these performance benefits, however, a key security vulnerability introduced by the relay is identified, and a Byzantine attack is formulated in which the relay injects structured, learned perturbations into its transmitted signal. The attack operates on intermediate communication representations subject to channel effects, with perturbations generated through a learned mapping applied at test time. It is shown to significantly degrade semantic inference accuracy at the destination. These results reveal a clear trade-off between SemCom gains and security risks in relay-assisted systems, motivating the need for secure relay designs that account for insider threats.