Optimization modeling is the process of translating real-world decision problems, often described in natural language, into formal mathematical formulations and executable solver code. While recent advances in large language models have shown promise in automating this process, many approaches focus on one-shot generation or rely on predefined execution-and-repair workflows, without explicitly learning a policy for tool use and revision. This stands in sharp contrast to real-world optimization modeling, which is inherently interactive and proceeds through repeated solve-debug-revise cycles. We introduce PEARL, a system for multi-turn solver-interactive optimization modeling that uses Python execution and mathematical programming solvers inside this loop. Rather than relying on a fixed repair workflow, PEARL learns when to test partial models, how to revise from solver diagnostics, and when to stop. It operates in a multi-turn tool-integrated setting where intermediate execution results, feasibility signals, and solution checks are used to improve both formulations and solver code before finalization. Across diverse optimization benchmarks, PEARL substantially improves executable objective agreement over the same-scale base model and is competitive with strong one-shot and tool-augmented baselines. Notably, PEARL-Qwen3-4B outperforms the much larger DeepSeek-V3.2-685B baseline under direct prompting on both macro- and micro-averaged Pass@1. Code and datasets are provided at https://github.com/AuroraLHL/PEARL.
Digital Twins support real-time decision-making through predictive models of physical systems, yet their reliability can deteriorate as operating conditions evolve. Maintaining predictive fidelity under such changes is challenging, particularly when model adaptation relies on limited streaming data, and updated models must be evaluated before deployment. Here, we present an online adaptation framework that integrates gradient-based drift detection, parameter-efficient fine-tuning, and statistical model validation within model predictive control. Changes in prediction-loss gradients trigger batch-wise model updates using Low-Rank Adaptation, while a paired moving-block bootstrap test evaluates predictive improvement before deploying the updated model. The framework is evaluated using a stochastic dynamical system with abrupt changes and a simulated directed energy deposition process with gradual changes in material properties. Results show improvements in predictive accuracy and uncertainty estimation during the evaluated post-adaptation intervals, alongside improved reference tracking and reduced constraint violations in the manufacturing case study compared with the investigated baselines. These findings demonstrate the potential of integrating continual validation with model adaptation while acknowledging the trade-off between responsiveness and predictive reliability in evolving Digital Twins.
The running-time distribution of a randomized algorithm is computed under independent randomness, but independence is rarely what a source guarantees. We study sources $\nu$ with $\nu[w]\le DP[w]^s$ for every finite prefix $w$ of the tape, where $P$ is an independent reference law, $0<s\le1$, and $D\ge1$; individual choices may be predictable, even determined by the past. The worst-case probability of any closed tape event $E$ is exactly $\min\{1,DC_s(E)\}$, where $C_s(E)$ is the infimum cost of covering $E$ by prefixes of varying depth. This tree max-flow/min-cut identity makes worst-case probabilities exact and optimizable. Three applications follow. In scheduling, $n$ tasks on one machine are each retried until the next random symbol lies in its success set; every schedule has the same makespan distribution under independent randomness, yet under our constraint two fixed orders miss a linear deadline with worst-case probabilities $1$ and $\exp[-\Theta(n)]$, and a static priority rule is optimal at every deadline. For the standard local walk on a $k$-CNF with an assignment satisfying a majority of each clause, we find the sharp source threshold $s_*$: below it, on some instances one admissible source forces nontermination for every clause selection; at it, with fixed $D$, the expected number of flips is at most cubic in the initial distance, against linear under independence, with matching instances at distances up to a constant fraction of the number of variables; a fixed margin above it restores linear dependence. For noisy predecessor search with prefix-bounded comparison errors, verified backtracking attains any prescribed error probability at or above a source threshold whose sharpness we leave open, with the same cubic-to-linear query bounds. Restarting on the shared tape restores exponential tails exactly when the nontermination event has a prefix cover of cost below one.
LLM agents can expose private local context (e.g., paths, emails) and credentials (e.g., API keys) as agent observations (e.g., tool outputs, shell logs, and file reads) are appended to provider-bound transcripts. Existing placeholder redaction is brittle: it can miss embedded or cross-turn references, over-redact benign lookalikes, and destroy the structure useful for reasoning. We present SlotGuard, a local transcript boundary that can hide sensitive data while retaining agents' performance. SlotGuard rewrites structural bindings as typed, suffix-aware slots, replaces secrets with format-preserving synthetic values, links cross-turn references with a lightweight session graph, and restores raw values only inside the trusted runtime. On controlled repository-oriented agent transcripts, SlotGuard removes all 20,814 annotated structurally sensitive characters across 9,229 paths and reduces credential leakage to 0.0% across 852 planted values. It remains close to raw-transcript task success across four upstream models, while generic redaction drops to 2.5%. Deterministic transcript rewriting takes a median of 14.424 microseconds per agent turn, excluding the optional semantic detector. The code is publicly accessible at https://github.com/illinoisdata/SlotGuard.
Grounded long-video question answering (Grounded LVQA) requires answering a question about a long video while localizing the short evidence interval that supports the answer. Recent agentic methods frame this task as multi-turn exploration with a single crop_video(start, end) action, which supports coarse-to-fine narrowing but provides no primitive for fine-to-coarse backtracking. As a result, these agents typically converge prematurely and cannot recover from an early mistake. We propose VideoTreeSearch (VTS), a framework that casts grounded LVQA as iterative self-correcting search over an adaptive temporal tree. VTS constructs a non-uniform tree through adaptive temporal segmentation so that each node corresponds to a semantically coherent segment, and trains an agent to navigate the tree through four discrete operations: zoom_in, zoom_out, shift, and answer. These operations expose backtracking and recovery as explicit, learnable primitives rather than implicit behaviors. To train this navigation, we introduce a trajectory synthesis pipeline that produces multi-step paths through the tree, including deliberate detours into incorrect branches followed by recovery. We use these trajectories for supervised fine-tuning, followed by reinforcement learning with grounding and answer-accuracy rewards. On three Grounded LVQA benchmarks (CG-Bench, Haystack-LVBench, Haystack-Ego4D), VTS outperforms the strongest prior multi-turn cropping agents by +12.5 mIoU on CG-Bench and +7.4 T-F1 on Haystack-Ego4D. The learned policy also transfers to general long-video QA, surpassing all prior multi-turn cropping agents on Video-MME, MLVU, and LVBench by up to +7.1 accuracy points. Ablations confirm that self-correcting hierarchical search is the central mechanism behind these gains: removing either adaptive descent or explicit backtracking substantially degrades performance.
Current evaluation of epistemic uncertainty relies on tasks such as out-of-distribution detection, active learning and selective prediction. However, common epistemic uncertainty scores, such as mutual information or the variance of the predictions, estimate the regret, the excess error of a predictor over the Bayes-optimal predictor. Building on the epistemic reject-option framework, we evaluate epistemic uncertainty using its ability to identify regret, and compare this evaluation with the other tasks. Benchmarking standard methods on datasets with repeated human labels reveals that the tasks rank the methods differently, including pairwise rank inversions between methods that are top-ranked on one criterion and bottom-ranked on another. The disagreement is present already at the optimum, where each task calls for a different score. Formulating selective prediction as a constrained optimization over coverage, expected risk, and regret, we prove that the optimal score is a convex combination of the ground-truth aleatoric and epistemic uncertainties, so that selective prediction and the regret task are the two endpoints of one family of tasks. Out-of-distribution detection, active learning and the regret task thus disagree, theoretically as well as empirically, on which method is a good estimator of epistemic uncertainty, while selective prediction departs from the regret task where the true risk and the regret order the inputs differently. Researchers must therefore state the downstream decision that an epistemic uncertainty estimate is meant to support, instead of relying on a general notion of model ignorance.
Multi-modal Large Language Models (MLLMs) have made substantial advances on benchmarks, yet their real-world effectiveness remains uncertain. This gap stems from the fundamental misalignment between benchmarks in controlled, static settings and the dynamic, interactive, and contextualized nature of real-world applications. To bridge this gap, we propose CEDI (Contextualized Evaluation of MLLMs through Dynamic, multi-round Interactions), a framework that recasts evaluation as a three-party interaction between an evaluatee model, an automated examiner, and a grader. The examiner conducts multi-turn, semi-structured conversations guided by a graph-based representation of the task. By navigating state-space transitions, CEDI deploys diverse strategies, from clarification requests to adversarial probes, to elicit performance evidence. We apply CEDI to visual hallucinations. Empirical results across multiple models, diverse settings, datasets, and domains show that contextualized, interactive evaluations reveal not only significantly more hallucinations than conventional static evaluation but also ones that more closely resemble those arising in practical use cases. We further show that hallucinations often accumulate over long contexts, through self-reinforcing dialogue history, and models are particularly vulnerable to questions requiring premise rejection or refusal. Together, these findings highlight CEDI as a step toward realistic, systematic, and ecologically valid assessments of MLLMs' capabilities. Code is available at github.com/williamium3000/cedi.
Human-in-the-loop Reinforcement Learning has become a popular approach for training, finetuning, and aligning robot behavior with user preferences. Our paper explores the feasibility of using brain signals via functional near-infrared spectroscopy (fNIRS) to modulate robot learning in simulation. We compare agents trained on passive (observational) versus active (demonstrative) interaction tasks, and test multiple methods for enhancing the RL algorithm with the neural signal, focusing on parameter augmentation in contrast to replacement. We further examine how model granularity and noise affect agent learning. Our results show that this framework is effective. The neural signal improves learning when augmenting trajectory priorities and state-action q-targets. Additionally, the framework learns successfully from offline data, offering a practical alternative for settings where real-time BCI setups are impractical or only limited data is available.
In online streaming video understanding, a video stream continues to arrive and queries may be issued at any time. Because streaming frames grow without bound, the system must continuously compress and retain information from the observed video prefix while future frames and future queries remain unknown. The core challenge is deciding what information to retain and how to organize the maintained history: as this history grows with the stream, memory cost increases and many redundant visual details are retained, whereas later queries often depend on specific entities, actions, and their temporal changes. To address this challenge, we introduce FOLIO, a training-free focused semantic memory system that records important parts of the stream in higher detail while keeping surrounding context compact. As the stream arrives, FOLIO updates memory at the segment level, guided by a dynamic focus state, combining a short-term visual buffer with a long-term semantic memory organized around observed entities and linked to a visual-evidence cache. At query time, lightweight hybrid retrieval combines direct matching over the structured memory with semantic query expansion. FOLIO achieves state-of-the-art performance, reaching 82.0/69.1 Perception/Backward accuracy on OVO-Bench with Qwen3-VL-8B and 74.5 overall accuracy on StreamingBench, while substantially reducing the cost of maintaining streaming memory by reserving detailed records for focused entities and storing surrounding context compactly.
Small molecules, crystals, and proteins all reduce to atoms in 3D space, yet their generative pipelines remain fragmented across domains, each with its Small molecules, crystals, and proteins all reduce to atoms in 3D space, yet their generative pipelines remain fragmented across domains, each with its own graph, equivariant, or frame-based architecture. Cross-domain training would mitigate per-domain data scarcity, but direct generation in 3D coordinate space cannot easily handle the heterogeneous structural priors of all three domains, and no prior latent autoencoder is simultaneously lossless and architecturally general across all three. We introduce SinAE, a single-architecture flow-matching autoencoder for molecules, crystals, and proteins, with vanilla Transformer encoder and decoder and no equivariant, graph, or domain-specific operators. Rather than requiring the encoder to capture fine-grained geometry, SinAE shifts the reconstruction burden into an iterative flow-matching decoder, achieving near-lossless reconstruction across domains and reducing reconstruction errors by orders of magnitude relative to prior latent baselines. The same per-token latent supports a standard Diffusion Transformer prior that reaches strong performance on molecular, crystal, and protein generation benchmarks. Joint molecule--crystal training strictly improves both domains, providing direct evidence of cross-domain transfer through a shared atomic latent. Code is available at https://github.com/BlueWhaleLab/SinAE .
Indexer-TopK, which computes scores and selects the top-$k$ candidates, is a key primitive in long-context sparse attention for large language models, vector retrieval for recommendation systems, and vector databases. However, existing GPU-based Indexer-TopK kernels like DeepSeek Sparse Attention (DSA) typically separate scoring from selection and materialize the full score matrix in high-bandwidth memory. This strategy is inefficient for large score matrices, such as those encountered in long-context sparse attention, due to excessive global-memory traffic, costly synchronization, and prohibitive memory overhead. In this study, motivated by the curse of dimensionality, we observe that sparse attention scores exhibit a score concentration phenomenon, with scores falling within a narrow range. Based on this observation, we propose LiteTopK, a sample-filter-select framework that estimates query-data score ranges from a small subset of the data. It uses these estimates to filter scores as they are computed, materializes only the candidates that pass the filter, and selects from the surviving candidates. In particular, we develop two implementations of LiteTopK. The first uses a conservative threshold estimate, while the second pairs a predicted threshold with a fallback strategy. Building on LiteTopK, we further propose LiteDSA, which reuses overlapping key/value entries across neighboring queries to accelerate attention computation. We integrate LiteTopK into vLLM, SGLang, and Megatron-LM. Our evaluation on eight B200 GPUs demonstrates prefill speedups of up to 1.49$\times$ across GLM-5.2, DeepSeek-V4-Flash, and Hy-4-Preview at a context length of 1M tokens, without compromising model capabilities.
In this paper, we propose SpectraReward, a training-free reward function that turns pretrained MLLMs into off-the-shelf reward models for image-generation reinforcement learning. Instead of asking the MLLM to judge a generated image or answer decomposed verification questions, SpectraReward measures how well the original prompt can be recovered from the generated image through a single image-conditioned, teacher-forced forward pass. We use the average image-conditioned prompt log-likelihood as the reward, directly reusing the MLLM's pretrained image-text alignment ability without preference labels, reward-model fine-tuning. We further introduce Self-SpectraReward, a special case for unified multimodal models where the policy's own understanding branch serves as the reward model for its generation branch, forming a closed-loop self-improving framework without external reward models or external knowledge. Extensive experiments validate SpectraReward through a broad image-generation RL study covering two diffusion models, three RL algorithms, nine reward MLLM backbones from four MLLM families spanning 4B to 235B parameters, and five out-of-distribution text-to-image benchmarks. Results show that both SpectraReward and Self-SpectraReward significantly and consistently improve generation performance and outperform prior MLLM-derived reward training methods. Further analysis reveals that larger reward MLLMs are not always better, while Self-SpectraReward can match or surpass much larger external reward models, suggesting that reward-policy alignment is a key factor for effective image-generation RL. Project Page: https://huangrh99.github.io/SpectraReward/
In multi-objective combinatorial optimization, unsupported non-dominated points typically outnumber supported points and are often significantly more challenging to compute. Recent studies show that extreme supported non-dominated points provide high-quality representations of the non-dominated set for certain binary problems. We demonstrate that this observation does not generalize to capacitated network optimization problems: representation quality decreases with increasing arc capacities, whereas supported non-dominated points consistently provide high-quality representations with respect to several quality indicators. However, supported point sets may still be too large in practical applications, where only a small, fixed number of alternatives is typically desired. Selecting fixed-size representations from the non-dominated set requires its computationally expensive generation and thus diminishes the computational advantages that representations are intended to provide. We therefore suggest the (extreme) supported points as alternative candidate sets in subset selection problems. Our numerical results show that restricting the candidate set to supported non-dominated points yields fixed-size representations of nearly the same quality as those selected from the complete non-dominated set. Overall, supported non-dominated points serve both as high-quality representations and as reasonable candidate sets for subset selection.
Modern large language models (LLMs) operate in interactive multi-turn settings, making multi-turn jailbreaking a realistic threat model and an important setting for automated red teaming. A core challenge in learning multi-turn jailbreak attackers is credit assignment: different turns contribute differently to the final outcome, yet existing learning signals are often too coarse to identify their individual contributions. We propose decomposed credit GRPO (DC-GRPO), a unified turn-level credit assignment framework for Group Relative Policy Optimization in multi-turn jailbreak learning. DC-GRPO assigns a separate group-relative learning signal to each turn by combining immediate and future credit, avoiding the credit misassignment induced by broadcasting a single trajectory-level score across the dialogue. We instantiate this framework with static and dynamic weighting rules that differ in how the two credit sources are balanced while sharing the same turn-level structure. Across four victim LLMs and three benchmarks, DC-GRPO achieves up to 98.26% average ASR_5@3, substantially outperforming the state-of-the-art methods, including SEMA (86.58%) and TROJail (86.23%), demonstrating the effectiveness of our turn-level group-relative credit assignment. Warning: This paper contains examples of harmful content.
As large language models (LLMs) generate and transform content at scale, reliable attribution becomes an increasingly important requirement for big data provenance. Although numerous watermarking methods have been proposed, existing evaluations predominantly emphasize sample-level detectability and robustness, leaving deployment requirements such as serving efficiency, population-scale false positives, multi-key attribution, and long-term verification less systematically examined. This paper presents a deployment-oriented systematization of LLM watermarking through the five Vs of big data: Volume, Velocity, Variety, Veracity, and Value. We classify representative watermarking methods across four dimensions: insertion point, verification authority, operational state, and adversarial threat model. We further map these methods to four deployment workloads: online generation, streaming detection, transformation pipelines, and ecosystem governance. Our synthesis identifies recurring limitations, including dependencies on generation-time access, additional inference overhead, sensitivity to downstream transformations, and incomplete reporting of calibration and lifecycle compatibility. Based on these findings, we propose workload-specific evaluation requirements covering statistical reliability, throughput, utility, robustness, and governance. This framework connects watermark design choices to practical deployment constraints and provides guidance for selecting and evaluating provenance mechanisms in large-scale data ecosystems.
Many real-world processes can be represented as compositions of functions along a directed acyclic graph (DAG). In causal modelling, these correspond to the underlying mechanisms; in engineering, to multiple fidelity levels; and in gene-regulatory networks, to transcription factors. These functions are partially observed across the DAG, with noisy and heterogeneously sampled measurements, posing significant challenges for reconstruction, uncertainty propagation, and inference. To tackle these challenges, we place priors over functions and naturally arrive at Deep Gaussian Processes over DAGs. We theoretically study their prior-collapse behaviour and the effect of graph topology and intermediate observations on the preservation of information. We obtain almost-sure lower bounds on the asymptotic frequency of depths at which the distinction between inputs is preserved, identify broad kernel classes for which these hold, and prove an observation by \citep{dunlop2018} on the role of input connections. We offer a structured variational approximation that retains graph dependencies, preserves compositional uncertainty, and captures the explaining-away behaviour of colliders. Finally, we empirically validate our theory and methodology at scale and under DAG misspecification. We then model a latent-collider DAG, a protein signalling network, and a multi-fidelity \mbox{heavy-ion} collision emulation task, attaining state-of-the-art performance while recovering low-fidelity contributions and yielding interpretability over the simulator hierarchy.
An AI agent may keep a successful check in context after an edit has invalidated its evidence. We give a semantics for specifying when a system may rely on a message and act on it. Reliability checks an asserted claim against accepted domain knowledge, sources, and messages in the selected context. Justified reliance additionally checks current evidence, authority, and the proposed world change. Relational component semantics and event traces represent nondeterministic outputs, iteration, and external changes. We characterise when merging claims preserves derivable conclusion classes, then construct the coarsest permitted quotient whose rules preserve the exact statuses of designated concrete claims. Construction is polynomial for unary rules; deciding even one permitted merge is coNP-complete for acyclic binary rules. This applies completeness and strong-preservation methods to the execution of quotiented rules. We also relate transition safeguards to runtime enforcement. Relative to a supplied set of possible worlds, a sound guard accepts an observation only when every matching world passes the checks. Exact acceptance requires observations that distinguish opposite-decision worlds; choosing sufficient services is NP-complete on explicit finite tables. A complete finite passport-renewal instance, Lean theorem encodings, Z3 instance checks, and an OCaml harness exercise the semantics. Domain knowledge, interpretation, source licenses, and policy remain application inputs.
Rich perceptual structure emerges in modern vision encoders. Several studies have shown that the internal representation in many vision transformers separates foreground objects from the background, however, little is known about the visual cues that have been learned by these models. In this work, we study whether figure-ground organization in a broad range of 25 vision transformers relies on generalizable shape cues, rather than on superficial texture statistics. To do so we train linear probes to classify patches as figure or ground, and evaluate them zero-shot on synthetic stimuli where individual cues are systematically manipulated. For a broad range of vision encoders, we find that probes are robust to the random replacement of textures in the input image, suggesting that the emerging figure-ground organization relies on shape cues. Probes also generalize in a zero-shot fashion to synthetic images where individual shape cues, recognizability and spatial priors are controlled for. Our experiments reveal a strong influence of surroundedness and convexity cues, while results for parallelism and symmetry are more nuanced. More detailed analyses reveal stronger evidence for shape-based cues in larger models, that models trained for object-recognition tend to be less robust to texture changes and that figure-ground information emerges most prominently in middle layers. Taken together, our study demonstrates that emerging figure-ground organization in recent vision encoders relies strongly on generalizable shape cues, advancing our understanding of the internal processing in vision transformers, and opening an exciting avenue for computational research on human visual perception. Code and data is available at https://github.com/mtangemann/mlvbench.
Exploration is essential to reinforcement learning (RL) since a policy cannot improve by repeatedly sampling the behaviors it already prefers. Standard methods inject stochasticity in the action space, but such jitter only yields rollouts close to the original action distribution. Escaping a weak policy often requires global perturbations that action noise cannot produce. Large language models (LLMs) and vision-language-action (VLA) models offer a pathway: they condition the policy on a natural language prompt, and since the rollout follows from it, modifying the prompt induces global changes. The challenge is finding prompts that induce useful global changes. With a weak policy that rarely succeeds, reward is too sparse to select on. Our idea is to refine prompts from the rollouts themselves: a vision-language model (VLM) reasons over the rollout video, diagnoses how the policy responded, and rewrites the prompt to elicit better behavior next time. Sampling a prompt and holding it fixed throughout a rollout provides episode-level commitment, analogous to posterior-sampling exploration. We call this strategy Prompt Driven Exploration (PDE). Across manipulation and reasoning tasks, PDE enables RL to learn successful policies even from zero-reward starts, and improves sample efficiency more broadly. Our website is available at https://xinyunsunshine.github.io/prompt-rl.
On-policy self-distillation (OPSD) is a promising approach to recursive self-improvement. In this setting, a model acts as its own teacher when augmented with privileged information, such as a solution to a math problem. Thinking models seem especially well positioned to teach themselves this way, since they can deliberate, verify, and self-correct. Surprisingly, we show that OPSD degrades thinking models on long reasoning traces: across five Qwen3 and OLMo thinking models evaluated on AIME24, AIME25, and HMMT25, privileged-context distillation causes a relative drop of up to 17\% in avg@16 accuracy. The degradation scales with the amount of privileged context withheld from the student and is most pronounced at long rollout budgets, where thinking models otherwise obtain their largest gains. This failure mode extends to distillation from a larger, privileged teacher: on-policy distillation (OPD) improves thinking models, but privileged OPD reverses these gains. Our diagnostics suggest that privileged teacher context reshapes learning at high-entropy forking positions, i.e., rollout positions where multiple continuations remain plausible and may lead to different reasoning paths. Privileged context lowers fork rates in thinking-model rollouts but not in instruction-tuned model rollouts, which may explain the observation that OPSD training helps instruct models more reliably than thinking models.At self-correction branches, privileged OPD penalizes sampled reconsideration tokens that vanilla OPD supports. Consistent with this, thinking models trained with a privileged teacher produce fewer verification, backtracking, and hedging markers, even after length normalization. These findings indicate that applying self-distillation methods to strong thinking models requires further consideration of token-level signal---especially around tokens related to correction and crucial reasoning steps.