We study minimal attention-only transformers under all-token corruption and show they admit a two-stage empirical Bayes interpretation. A single attention step computes a kernel-weighted posterior mean with respect to the empirical distribution defined by the context. Depth refines this distribution through particle dynamics (Stage 1), while a long-range skip-connection carries the noisy input as a query for posterior inference (Stage 2), revealing distinct statistical roles for depth and attention residuals. The framework isolates a minimal setting in which the context itself induces a depth-dependent energy landscape governing in-context inference. We show that effective denoising can emerge without an explicit noise schedule: a fixed kernel bandwidth and finite integration horizon suffice, yielding a principled depth-noise relationship. We further establish a posterior-mean recovery guarantee for Gaussian priors, showing the empirical estimator converges to the Bayes-optimal predictor under sequential asymptotic conditions. Connecting these dynamics to reverse-diffusion limits, our results provide a statistical interpretation of attention as in-context inference via sample-based posterior estimation, without explicit density modeling.
Task success can hide process anomalies in real-world agent executions. An agent may pass the final task oracle while still accumulating unresolved ambiguity, unsafe external writes, ignored errors, weakly grounded commitments, or capability-boundary overcommitment. We study this mismatch as the Outcome-Process Gap and introduce OpenClawBench, a large-scale dataset for measuring and supervising process-side anomalies in real agent execution processes. OpenClawBench is built from BFCL-driven OpenClaw sessions produced by 6 source models and contains 31,264 annotated trajectories. FullTax converts the aligned trajectories into structured anomaly supervision: binary labels, supporting evidence, onset/span localization, severity, recoverability, and a 5-class anomaly taxonomy. Using OpenClawBench, we make the Outcome-Process Gap measurable. Among 31,135 oracle-passing executions, 2,904 (9.33%) are still labeled process-anomalous under FullTax. Within oracle-passing executions that contain high-risk process evidence, 1,765 of 1,904 (92.70%) are labeled anomalous. On Gold-300, a test subset labeled independently by two human annotators, silver labels match the final human binary label on 93.3% of trajectories. A LoRA-fine-tuned Gemma 3 12B detector trained on the high-confidence FullTax supervised pool reaches binary F1=0.729 on the cleaner-labels silver test split (n=2,646). It outperforms the GPT-5.4 frontier reference by +0.302 absolute, the no-fine-tuning base by +0.357, and wins on all six source-model agent slices. The gain comes from calibration rather than higher recall: zero-shot detectors over-flag at 42-50% against a 14.8% label rate, while the fine-tuned detector predicts anomalies 18.2% of the time. Together, OpenClawBench turns real agent execution logs into auditable and reusable supervision for studying, diagnosing, and operationally monitoring runtime agent reliability.
A model trained with an imperfect reward in RL may learn to exploit the reward with undesired behavior. We study a controlled form of this problem in competitive programming. A program may pass small tests while violating the intended efficiency requirement on larger inputs. We construct a family of rewards that progressively relax the efficiency constraint, making the reward increasingly easy to exploit, and train separate policies. This sweep reveals a correctness-efficiency frontier. Relaxing the reward leads to less efficient programs and more resource-limit failures. Stricter rewards suppress these failures but increase correctness failures near the model's capability boundary. This frontier is also reflected in weight space. Interpolating between the resulting models reproduces the same behavioral change. More surprisingly, extrapolation in both directions further amplifies or suppresses it beyond the trained checkpoints. Extensive experiments show that this pattern holds across pure reasoning, tool use, and agentic coding, and at both 32B and 7B scales. This suggests a simple intervention for this form of reward hacking: use a loose reward to amplify the undesired behavior, use the weight-space difference to a strict-reward model as its steering vector, and then extrapolate past the strict model to suppress it.
Language models are increasingly used to make decisions about people, such as whom to interview, which insurance claims to approve and which patients to see first. Research on bias in these decisions has focused on whether a person's race or gender changes the outcome. However, developers also write text that the model reads before every decision, including a persona that gives the model a role and often names a project, team or office. Whether such text affects whom a model selects remains largely unexamined. We asked what happens when a string in the persona also appears in one candidate's record. Across four kinds of decision, fourteen publicly released models and 328,480 choices between two candidates, instruction-tuned models, the kind used in chat assistants, selected the candidate whose record repeated the persona's string in 71.3% of decisions between equally qualified candidates, compared with 50.1% when no string was shared. When that candidate was one step less qualified, the rates were 31.5% and 11.7%. A made-up project code was enough. The effect persisted when the record labelled the string a conflict of interest, even in models that otherwise selected such a candidate less often. When the shared string was the candidate's race, a similar shift appeared. Bias audits built around protected attributes can at most show the resulting gaps between groups, not their source.
Certified inference-time copyright defences publish a divergence budget between the served model and a safe anchor trained without the protected works. We ask what such a budget certifies. Proved: a budget of K nats says nothing about a work whose surprisal under the anchor is at most K, at every R\'enyi order. Anchored decoding charges per token, so its budget grows with the output. Measured: at its authors' setting, the budget exceeds the anchor's surprisal of every one of the 758 protected passages we test, so it certifies nothing about them. Proved: serving one of n anchor draws, by any rule, is certified at log n nats for the whole output, at any length; in installments, a window is certified only given the text up to a block boundary. Measured: over a vetted anchor, with an adversarial scorer, selection reproduced no protected passage near-verbatim. Its draws cost 21.8x a metered decode on an anchor-only server. Judged: in the headline configuration, at the only published meter budget that bounds a 50-token window, selection's gain over one anchor draw is reliably positive and the meter's is, by a post-hoc equivalence test, zero. At matched compute selection loses. At the authors' sampling settings on the 8B, the two tie at that budget. Where the meter's certificate is vacuous, the ranking depends on how the stronger model is served. It reverses under a chat template, at the authors' sampling settings on the 8B, and on AlpacaEval.
Single-axis mitigations of reward-model biases, such as reducing proxy reliance on length, sycophancy, or style, can redirect optimization pressure onto correlated proxies rather than eliminate it, a new type of reward hacking we call reward bias substitution. The failure is enabled by a measurement-versus-optimization gap between mitigation evaluation on the audit distribution and policy training on the policy-induced distribution. Prior reward-hacking analyses assume a fixed reward proxy, whereas we study the mitigation operator that removes one bias from it, the basic step of iterative reward refinement. We formalize mitigation outcomes into a regime taxonomy and prove that successful mitigation, bias substitution, and overcorrection can produce identical observables under every audit-distribution score, including ranking accuracy and win-rate, even when granted oracle access to the true reward. Across 30 published preference-learning mitigation works, no method we survey reports the evidence needed to certify successful mitigation. Augmenting evaluation with policy-induced distributions while tracking multiple biases provably resolves this distributional gap, and we translate this into actionable prescriptions for mitigation methods and benchmarks. We demonstrate bias substitution in RLHF, where a length penalty in GRPO shortens responses as intended yet makes the policy overconfident and less accurate on open-ended questions, show that the same distributional gap hides failures of published mitigations, and find the necessary preconditions for reward bias substitution documented in published results and in our study of length-sycophancy coupling in LLM responses.
Gradient-flow optimization is usually viewed as an algorithmic procedure for minimizing empirical loss, with training duration selected by validation or heuristic early stopping rules. We develop a statistical inference framework for gradient-flow training. We show that whenever fitted values evolve through a time-invariant positive semidefinite training operator, the output at each time is equivalent to the best linear unbiased predictor under a corresponding random-effects model. Training time then becomes a variance-component parameter governing variance reallocation from residual noise to structured signal. This turns two training decisions into inferential problems: whether training is needed becomes a variance-component test for signal beyond initialization, and how long to train becomes restricted maximum likelihood (REML) estimation of the training-time variance component. We show that the REML-guided early stopping rule selects the time at which optimized spectral losses become decorrelated from the training-operator eigenvalues. The asymptotic prediction optimality of the REML-guided early stopping time is established for fixed-design in-sample risk and random-design out-of-sample risk. Deep learning models in fixed-kernel gradient regimes provide canonical instantiations for our results. Numerical experiments and a UK Biobank proteomics application show competitive accuracy of the REML-guided early stopping time with reduced reliance on validation splits and repeated checkpoint evaluation.
Backpropagation is the default learning rule for artificial neural networks and is often treated as the settled approach whenever differentiability is available. In this work, we revisit this convention through a theoretical lens of sample efficiency. We introduce a unified vectorized feedback framework for loss-based and reward-based learning on computational graphs, in which synthetic gradients emerge as a natural alternative to backpropagation. We characterize the conditions under which synthetic gradients can achieve a lower gradient-estimation mean squared error than backpropagation. We construct examples illustrating that this sample efficiency advantage can be arbitrarily large. Experiments on contextual bandits and reinforcement learning tasks demonstrate the potential of our theoretical findings.
Humans know when to reach for help e.g. $347 \times 28$ warrants a calculator while $2+2$ does not. Language models do not. Prompt-based approaches can instruct a model when to invoke tools, but this scaffolding does not teach it to recognize the boundary of its own knowledge. RL approaches that assign a single outcome reward to the whole trajectory fare no better: trajectory-level credit cannot isolate which tool call in a successful episode actually helped, nor penalize unnecessary calls. We propose CARL (Competence-Aware Reinforcement Learning), which trains a critic on the model's own rollouts to learn where parametric knowledge suffices and where it needs external help. By decomposing each rollout at natural tool-use boundaries (e.g., code fence delimiters and context block transitions), CARL assigns independent credit to each segment from a single binary outcome, without external judges or step-level annotations. As a result, erroneous tool calls, incorrect extractions, and unnecessary calls each receive appropriately signed advantages. The trained critic captures the model's domain competence: it separates parametrically solvable from tool-dependent questions with AUC 0.93 at 7B. On five benchmarks spanning arithmetic, multi-hop factual QA, and numerical reasoning over financial tables, CARL improves average exact-match accuracy on the three search benchmarks by 6.7 points at 7B and 9.6 points at 3B over the best RL baseline, with the largest gain (+8.3 EM at 7B, +12.5 EM at 3B) on MuSiQue. At 7B, CARL has a 53% lower tool-use rate than Search-R1 PPO on parametrically answerable questions while remaining 9.8 EM points more accurate on them. Gains are largest at small scale: the 3B improvement is $1.4\times$ the 7B improvement, suggesting that knowing when to ask disproportionately benefits models with smaller parametric memory.
We study the query complexity of sampling from high-dimensional Gaussian distributions using gradient information. In the standard oracle model, exact gradients expose only matrix-vector products with the precision matrix, leading to polynomial approximation barriers and a characteristic $\sqrt{\kappa}$ dependence on the known spectral range. We show that this barrier disappears when the sampler is allowed to query smoothed scores, namely gradients of the logarithms of the Gaussian-convolved densities. For a centered Gaussian target with precision matrix $\Lambda$, a smoothed-score query at noise level $\tau$ gives access to the resolvent $(\Lambda+\tau^{-1}I)^{-1}$. Combining geometrically spaced noise levels with sinc-quadrature rational approximation, we obtain a sampler with $q=O((\log\kappa+\log(e\sqrt d/\delta_{\rm TV}))\log(e\sqrt d/\delta_{\rm TV}))$ smoothed-score queries for total variation error $\delta_{\rm TV}$, improving the condition-number dependence from $\sqrt{\kappa}$ to logarithmic. We also study finite-bit gradient oracles. Using coordinatewise quantization of the transformed smoothed-score answers and a final dithering step, we obtain a sampling scheme whose total communicated gradient information is polylogarithmic in $\kappa$; in particular, for fixed dimension and accuracy, the bit complexity is $O(\log^2\kappa)$. To complement these upper bounds, we introduce a channel-synthesis, or reverse-Shannon, converse technique for sampling lower bounds. This converts total-variation simulation guarantees into communication requirements and yields an $\Omega(d\log\kappa)$ bit lower bound for $d\ge2$ and fixed total variation error below one, as $\kappa\to\infty$. Together, these results identify smoothed scores as a provably more informative oracle for sampling and give nearly matching upper and lower bounds for its finite-bit complexity.
Generative models, from diffusion models to large language models, achieve remarkable performance but at a cost in training data orders of magnitude larger than what biological learners require. An alternative paradigm has emerged in which networks are trained to predict their own latent representations of related views or masked regions, as in data2vec and JEPA -- an idea related to predictive-coding accounts of the cortex. Despite strong empirical results, the theoretical understanding of these methods remains limited. Central questions include: by how much does latent prediction actually improve data efficiency? Is there a benefit to stacking such methods into multi-scale hierarchies? We answer both using as data a tractable probabilistic context-free grammar that captures the compositional structure of natural language and images. Such a grammar generates strings of visible tokens by recursively applying production rules along a tree of hidden symbols of depth $L$. For such data, supervised or token-level SSL require a number of samples exponential in $L$ to recover the latent tree; we prove that latent prediction achieves this with a number of samples constant in $L$, up to logarithmic factors. We confirm this bound with (i) a hierarchical clustering algorithm, (ii) an end-to-end neural network whose predictor-clusterer modules predict their own latents at each level via gradient descent, and (iii) the first sample-complexity analysis of data2vec, which we show implicitly performs hierarchical latent prediction. This suggests that explicit stacking such as H-JEPA is largely redundant.
3D shape completion from partial scans remains challenging for unseen categories and noisy real-world observations, where geometry alone is often insufficient for inferring missing structure. We present DinoComplete, a deterministic and efficient shape completion framework that augments geometric reconstruction with voxel-aligned semantic priors distilled from DINO features. First, we construct multi-view DINO feature volumes aligned with ShapeNet data and train a student network to predict dense semantic features directly from incomplete shapes. These predicted features capture global structure and part-aware semantic context while remaining aligned with the underlying geometry. We then integrate these distilled features into a completion network, where geometric and semantic voxel representations are fused through voxel state-space modeling. To enable efficient long-range reasoning without sacrificing resolution, we introduce a Multi-Scale Voxel Mamba module that refines the fused features by combining full-grid and chunk-wise sequence modeling. Experiments on unseen ShapeNet categories and ScanNet objects show that DinoComplete achieves competitive completion quality than prior deterministic and generative based completion methods while using relatively fewer parameters, requiring lower memory, and achieving faster inference. Our results demonstrate that distilling semantic priors from visual foundation models improves generalization and robustness in 3D shape completion.
Understanding human gaze behavior is essential for complex scene comprehension and human-computer interaction. Traditional gaze following models are typically restricted to pure spatial localization, lacking the high-level capacity to reason about semantic targets or complex social contexts. Furthermore, these models often process individuals sequentially, requiring redundant computations over the same scene image for multi-person inference. While recent Vision-Language Models (VLMs) offer the exceptional semantic reasoning needed to address gaze-related semantic tasks, their reliance on discrete text generation inherently limits precision in continuous spatial tasks like gaze localization. To bridge this gap, we propose OmniGF, a unified vision-language framework that adapts foundational VLMs for highly scalable multi-person gaze reasoning. The model adopts a dual-branch decoding strategy: a structured language branch generates discrete reasoning states, while a continuous spatial branch directly taps into the VLM's dense hidden states. Supervising these extracted representations with high-resolution gaze target heatmaps effectively overcomes the spatial bottleneck of text-only coordinate generation. Furthermore, to explicitly ground the model in multi-person scenes, we augment the input with head embeddings encoded from cropped head images, providing fine-grained appearance and orientation cues for all individuals simultaneously. By modeling all individuals and leveraging the strong semantic capability of VLMs, OmniGF seamlessly integrates precise spatial gaze target estimation, semantic gaze prediction, and complex social gaze reasoning. Extensive experiments demonstrate that our framework establishes new state-of-the-art performance across multiple standard benchmarks. Code is available at https://github.com/cvlab-stonybrook/omnigf.
A representation that scrambles the true degrees of freedom of the world cannot support reliable planning or compositional generalization. We prove that LeJEPA (alignment plus Gaussian regularization) linearly recovers the world's latent variables from nonlinear observations, a property known as linear identifiability, in a class of worlds where latents evolve under stationary, additive-noise transitions. Our main result is that among all such worlds, the Gaussian is the unique latent distribution for which this guarantee holds. The forward direction rests on a spectral decomposition in which each degree of nonlinearity is strictly penalized by alignment, making the linear map the optimum; the converse rules out every non-Gaussian alternative. We further prove an approximate identifiability result where the guarantee degrades gracefully, and show that linear, orthogonal identifiability enables optimal latent-space planning. We validate the theory with experiments ranging from 2D examples to 4096-dimensional latents, including ablations, pixel-based robotic control, and natural images from ImageNet-X. Our theory turns an empirically successful recipe into a mathematical guarantee, providing the foundation for building World Models that provably recover the structure of the world.
We study adversarial online learning with hidden-convex losses, i.e., nonconvex losses that become convex after a nonlinear reparameterization. Ghai, Lu and Hazan (2022) proved that, under geometric and smoothness assumptions, online gradient descent (OGD) on such nonconvex losses approximately simulates online mirror descent (OMD) on the underlying convex losses with a suitable regularizer, yielding $\mathcal{O}(T^{2/3})$ regret. They left open whether the optimal $\Theta(\sqrt{T})$ regret from online convex optimization can be recovered in this hidden-convex setting. We answer this question affirmatively. More specifically, via a sharper discrete-time algorithmic equivalence argument, we prove that OGD achieves $\mathcal{O}(\sqrt{T})$ regret under the same assumptions, matching the optimal worst-case rate for adversarial online convex optimization. We also address another open question of Ghai, Lu and Hazan (2022) by clarifying the geometry required for this algorithmic equivalence. We replace the diagonal-Jacobian sufficient condition with a necessary-and-sufficient Hessian compatibility condition, thereby expanding the class of admissible reparameterizations. We complement our tight regret bound with a lower bound showing that OGD suffers $\Omega(T)$ regret even with a stepsize of order $1/\sqrt{T}$, for a smooth Hessian incompatible reparameterization between convex domains and an adversarial sequence of hidden-convex losses. Finally, we extend our analysis to one-point bandit feedback and prove a $\mathcal{O}(T^{3/4})$ expected regret bound for bandit OGD with spherical smoothing, matching its classical rate on convex losses.
Turbulence is ubiquitous in engineering and science, yet direct simulation is prohibitively expensive. The Reynolds-averaged Navier-Stokes (RANS) equations provide savings exceeding ten orders of magnitude but introduce unclosed terms. Offline-trained machine-learning (ML) closures suffer distribution shift in predictive simulations, while ML methods that bypass the governing equations struggle to generalise from scarce high-fidelity data. We develop a physics-derived deep learning closure for RANS, the Deep Algebraic Reynolds Stress Model (DARSM), which can be trained on one or two flow cases and accurately generalise across an order of magnitude in Reynolds number, to unseen geometries, and across flow regimes. A neural network maps flow invariants to empirical parameters in an implicit algebraic Reynolds stress equation, derived from the Reynolds stress transport equations under the weak-equilibrium assumption, imposing physics-based structure on the ML closure. End-to-end optimisation through the governing PDEs eliminates distribution shift, but both unrolled and implicit automatic differentiation fail on the stiff coupled solver. We derive adjoint equations that exploit the solver's implicit-explicit structure for efficient optimisation. On canonical square-duct and periodic-hill benchmarks, DARSM reduces average test velocity error over baseline RANS by 2-4x across Reynolds number, geometries, and flow regimes, with peak case-level reductions of 12x. The model trained on attached, anisotropy-dominated flows (square duct) accurately generalises without retraining to separated flows (periodic hills), a regime change in the underlying physics. DARSM also outperforms established ML methods: offline training, tensor-basis neural networks, field-inversion machine learning, DeepONets, physics-informed neural networks, and an adjoint-trained source-term closure.
Long-lived AI agents are increasingly deployed as persistent operational systems, yet they are still mostly evaluated like freshly initialized models. Day-one benchmarks miss a basic systems question: how does an agent's reliability change after deployment? Even when model weights are frozen, an agent's effective state keeps changing as it compresses interaction history, retrieves from a growing memory store, revises facts after updates, and undergoes routine maintenance. Reliability therefore becomes a lifespan property of the full agent harness, not only a snapshot property of the base model. We introduce AgingBench, a longitudinal reliability benchmark for agent lifespan engineering: measuring not only whether deployed agents degrade, but what form the degradation takes and where repair should target. AgingBench organizes agent aging into four mechanisms: compression aging, where writing and rewriting drops future-relevant details; interference aging, where accumulated similar memories crowd out the target fact; revision aging, where changed or derived state is not updated correctly; and maintenance aging, where lifecycle events such as model swapping trigger regressions. To diagnose these failures, AgingBench uses temporal dependency graphs and counterfactual probes that produce diagnostic profiles for the write, retrieval, and utilization stages of the memory pipeline. Across 6 scenarios, 10 memory systems and 6 models from two families, we find that agent aging is not one-dimensional over 400 runs spanning 50-200 sessions: a well-formed reply can hide a value the memory no longer supplies; derived state collapses as updates accumulate; and the same wrong answer can require different repairs depending on what the diagnostic profile points to.
Proximity-inducing drugs, including PROTACs, represent a promising therapeutic modality. Their activity is driven by the geometry of the assembled ternary complex, which determines whether the target is ubiquitinated. Co-folding models perform poorly in these cases due to scarcity of training data and the dynamic nature of such complexes. Structural proteomics can supply the missing geometric information and steer the model towards an accurate prediction. However, current models accept only positive signals, while a differential structural mass-spectrometry experiment also reports negative signals. We introduce AIMS-Fold, an inference-time guidance layer built on Boltz-2 that requires no retraining and accepts repulsive constraints alongside attractive ones. We tested AIMS-Fold on 28 PROTAC, glue and antibody complexes that Boltz-2 fails to predict. From only five generated models, guidance recovers the correct conformation in 64% of cases. The gap does not close with scale: across 100 models and 20 seeds, unguided Boltz-2 reaches 14% while AIMS-Fold reaches 86%. Repulsive constraints alone recover the correct conformation on some systems, occasionally exceeding the matched attractive constraints. Taken together, these results demonstrate the strength of utilizing the full structural proteomics signal, positive and negative, to steer generative models toward the correct conformation, addressing a central problem in rational degrader design.
Audio-language models (ALMs) are increasingly used in real-world applications that require music understanding, including tutoring, captioning, recommendation, and production. Yet existing benchmarks rarely evaluate a fundamental component of this understanding: pitch hearing. Pitch underpins the identification of notes, chords, and melodies and the grounding of captions in musical events, but it remains unclear how reliably ALMs perceive fine-grained pitch across sound sources, acoustic conditions, and response formats. We introduce PitchBench, an evaluation suite comprising 28 tasks that assess absolute and relative pitch perception under variations in loudness, note duration, sound source, time stretching, background noise, and other acoustic conditions. Tasks range from identifying isolated pitches to tracking voices in four-part musical textures. Evaluating frontier ALMs, we find that pitch hearing remains highly unreliable. Models perform poorly across settings, with accuracy varying sharply depending on the notation or prompt format, indicating instruction drift. Specialized DSP and neural pitch baselines achieve mean scores of approximately 73.2% and 75.2%, respectively, compared with approximately 53.9% for the strongest ALM. We release PitchBench as an evaluation benchmark, a training dataset, and a Python package with data generation tools to support future research on pitch-aware audio-language modeling.
Subject-driven image generation aims to synthesize new images that preserve the identity of the given subject while following textual instructions. Existing approaches often encode text and reference images separately. This limits cross-modal reasoning abilities and causes copy-paste artifacts. Recent frameworks that connect multimodal models and diffusion models improve instruction following, but largely overlook identity preservation. To address these limitations, we condition diffusion models on Multimodal Large Language Models (MLLMs) that jointly encode text and reference images, and augment it with VAE-based identity conditioning. A novel Dual Layer Aggregation (DLA) module is designed to aggregate multi-level MLLM features for optimal conditioning, and a multi-stage denoising strategy is applied to progressively balance the semantic information from MLLM and fine-detail identity from VAE during inference. Extensive experiments demonstrate that our approach harmonizes multimodal understanding with identity preservation, mitigates copy-paste issues, and achieves superior performance regarding human preference on subject-driven image generation. Our project website is available at https://zsh2000.github.io/squeeze-mllm-subject-gen/.