The efficient computation of parametric solution sensitivities is a key challenge in the integration of learning-enhanced methods with nonlinear model predictive control, as their availability is crucial for many learning algorithms. This paper discusses the computation of solution sensitivities of general nonlinear programs using the implicit function theorem and smoothed optimality conditions treated in interior-point methods (IPM). We introduce sequential barrier quadratic programming (SBQP), which generalizes SQP with an IPM quadratic programming (QP) solver by allowing the barrier parameter to approach a prescribed nonnegative target value, recovering classical SQP when this target is zero. Previous works are limited to convex or unconstrained formulations, or lack an implementation for efficient sensitivity evaluation. The publication is accompanied by an efficient open-source implementation within the `acados` framework, providing both forward and adjoint sensitivities for general optimal control problems, achieving speedups exceeding 3x over the state-of-the-art solvers `mpc.pytorch` and `cvxpygen`.
This study investigates two models of varying complexity for optimizing intraday arbitrage energy trading of a battery energy storage system using a model predictive control approach. Scenarios reflecting different stages of the system's lifetime are analyzed. The findings demonstrate that the equivalent-circuit-model-based non-linear optimization model outperforms the simpler linear model by delivering more accurate predictions of energy losses and system capabilities. This enhanced accuracy enables improved operational strategies, resulting in increased roundtrip efficiency and revenue, particularly in systems with batteries exhibiting high internal resistance, such as second-life batteries. However, to fully leverage the model's benefits, it is essential to identify the correct parameters.
Heterogeneity in sensors and actuators across environments poses a significant challenge to building large-scale pre-trained world models on top of this low-dimensional sensor information. In this work, we explore pre-training world models for heterogeneous environments by addressing key transfer barriers in both data diversity and model flexibility. We introduce UniTraj, a unified dataset comprising over one million trajectories from 80 environments, designed to scale data while preserving critical diversity. Additionally, we propose TrajWorld, a novel architecture capable of flexibly handling varying sensor and actuator information and capturing environment dynamics in-context. Pre-training TrajWorld on UniTraj yields substantial gains in transition prediction, achieves a new state-of-the-art for off-policy evaluation, and also delivers superior online performance of model predictive control. To the best of our knowledge, this work, for the first time, demonstrates the transfer benefits of world models across heterogeneous and complex control environments. Code and data are available at https://github.com/thuml/TrajWorld.
Traditional motion planning methods for robots with many degrees-of-freedom, such as mobile manipulators, are often computationally prohibitive for real-world settings. In this paper, we propose a novel multi-model motion planning pipeline, termed Re4MPC, which computes trajectories using Nonlinear Model Predictive Control (NMPC). Re4MPC generates trajectories in a computationally efficient manner by reactively selecting the model, cost, and constraints of the NMPC problem depending on the complexity of the task and robot state. The policy for this reactive decision-making is learned via a Deep Reinforcement Learning (DRL) framework. We introduce a mathematical formulation to integrate NMPC into this DRL framework. To validate our methodology and design choices, we evaluate DRL training and test outcomes in a physics-based simulation involving a mobile manipulator. Experimental results demonstrate that Re4MPC is more computationally efficient and achieves higher success rates in reaching end-effector goals than the NMPC baseline, which computes whole-body trajectories without our learning mechanism.
We propose KnotGym, an interactive environment for complex, spatial reasoning and manipulation. KnotGym includes goal-oriented rope manipulation tasks with varying levels of complexity, all requiring acting from pure image observations. Tasks are defined along a clear and quantifiable axis of complexity based on the number of knot crossings, creating a natural generalization test. KnotGym has a simple observation space, allowing for scalable development, yet it highlights core challenges in integrating acute perception, spatial reasoning, and grounded manipulation. We evaluate methods of different classes, including model-based RL, model-predictive control, and chain-of-thought reasoning, and illustrate the challenges KnotGym presents. KnotGym is available at https://github.com/lil-lab/knotgym.
In this paper, we consider the design of model predictive control (MPC) algorithms based on deep operator neural networks (DeepONets). These neural networks are capable of accurately approximating real and complex valued solutions of continuous time nonlinear systems without relying on recurrent architectures. The DeepONet architecture is made up of two feedforward neural networks: the branch network, which encodes the input function space, and the trunk network, which represents dependencies on temporal variables or initial conditions. Utilizing the original DeepONet architecture as a predictor within MPC for Multi Input Multi Output (MIMO) systems requires multiple branch networks, to generate multi output predictions, one for each input. Moreover, to predict multiple time steps into the future, the network has to be evaluated multiple times. Motivated by this, we introduce a multi step DeepONet (MS-DeepONet) architecture that computes in one shot multi step predictions of system outputs from multi step input sequences, which is better suited for MPC. We prove that the MS DeepONet is a universal approximator in terms of multi step sequence prediction. Additionally, we develop automated hyper parameter selection strategies and implement MPC frameworks using both the standard DeepONet and the proposed MS DeepONet architectures in PyTorch. The implementation is publicly available on GitHub. Simulation results demonstrate that MS-DeepONet consistently outperforms the standard DeepONet in learning and predictive control tasks across several nonlinear benchmark systems: the van der Pol oscillator, the quadruple tank process, and a cart pendulum unstable system, where it successfully learns and executes multiple swing up and stabilization policies.
Configuration-Constrained Tube Model Predictive Control (CCTMPC) offers flexibility by using a polytopic parameterization of invariant sets and the optimization of an associated vertex control law. This flexibility, however, often demands computational trade-offs between set parameterization accuracy and optimization complexity. This paper proposes two innovations that help the user tackle this trade-off. First, a structured framework is proposed, which strategically limits optimization degrees of freedom, significantly reducing online computation time while retaining stability guarantees. This framework aligns with Homothetic Tube MPC (HTMPC) under maximal constraints. Second, a template refinement algorithm that iteratively solves quadratic programs is introduced to balance polytope complexity and conservatism. Simulation studies on an illustrative benchmark problem as well as a high-dimensional ten-state system demonstrate the approach's efficiency, achieving robust performance with minimal computational overhead. The results validate a practical pathway to leveraging CCTMPC's adaptability without sacrificing real-time viability.
This paper presents a comprehensive overview of data-driven model predictive control, highlighting state-of-the-art methodologies and their numerical implementation. The discussion begins with a brief review of conventional model predictive control (MPC), which discusses both linear MPC (LMPC) and nonlinear MPC (NMPC). This is followed by a section on data-driven LMPC, outlining fundamental concepts and the implementation of various approaches, including subspace predictive control and prediction error methods. Subsequently, the focus shifts to data-driven NMPC, emphasizing approaches based on recurrent neural network-based models. The paper concludes with a review of recent advancements in data-driven MPC and explores potential directions for future research.
Can we build accurate world models out of large language models (LLMs)? How can world models benefit LLM agents? The gap between the prior knowledge of LLMs and the specified environment's dynamics usually bottlenecks LLMs' performance as world models. To bridge the gap, we propose a training-free "world alignment" that learns an environment's symbolic knowledge complementary to LLMs. The symbolic knowledge covers action rules, knowledge graphs, and scene graphs, which are extracted by LLMs from exploration trajectories and encoded into executable codes to regulate LLM agents' policies. We further propose an RL-free, model-based agent "WALL-E 2.0" through the model-predictive control (MPC) framework. Unlike classical MPC requiring costly optimization on the fly, we adopt an LLM agent as an efficient look-ahead optimizer of future steps' actions by interacting with the neurosymbolic world model. While the LLM agent's strong heuristics make it an efficient planner in MPC, the quality of its planned actions is also secured by the accurate predictions of the aligned world model. They together considerably improve learning efficiency in a new environment. On open-world challenges in Mars (Minecraft like) and ALFWorld (embodied indoor environments), WALL-E 2.0 significantly outperforms existing methods, e.g., surpassing baselines in Mars by 16.1%-51.6% of success rate and by at least 61.7% in score. In ALFWorld, it achieves a new record 98% success rate after only 4 iterations.
In this paper, we study homothetic tube model predictive control (MPC) of discrete-time linear systems subject to bounded additive disturbance and mixed constraints on the state and input. Different from most existing work on robust MPC, we assume that the true disturbance set is unknown but a conservative surrogate is available a priori. Leveraging the real-time data, we develop an online learning algorithm to approximate the true disturbance set. This approximation and the corresponding constraints in the MPC optimisation are updated online using computationally convenient linear programs. We provide statistical gaps between the true and learned disturbance sets, based on which, probabilistic recursive feasibility of homothetic tube MPC problems is discussed. Numerical simulations are provided to demonstrate the efficacy of our proposed algorithm and compare with state-of-the-art MPC algorithms.
This paper proposes an integrated approach for the safe and efficient control of mobile robots in dynamic and uncertain environments. The approach consists of two key steps: one-shot multimodal motion prediction to anticipate motions of dynamic obstacles and model predictive control to incorporate these predictions into the motion planning process. Motion prediction is driven by an energy-based neural network that generates high-resolution, multi-step predictions in a single operation. The prediction outcomes are further utilized to create geometric shapes formulated as mathematical constraints. Instead of treating each dynamic obstacle individually, predicted obstacles are grouped by proximity in an unsupervised way to improve performance and efficiency. The overall collision-free navigation is handled by model predictive control with a specific design for proactive dynamic obstacle avoidance. The proposed approach allows mobile robots to navigate effectively in dynamic environments. Its performance is accessed across various scenarios that represent typical warehouse settings. The results demonstrate that the proposed approach outperforms other existing dynamic obstacle avoidance methods.
Sampling-based model predictive control (MPC) offers strong performance in nonlinear and contact-rich robotic tasks, yet often suffers from poor exploration due to locally greedy sampling schemes. We propose \emph{Model Tensor Planning} (MTP), a novel sampling-based MPC framework that introduces high-entropy control trajectory generation through structured tensor sampling. By sampling over randomized multipartite graphs and interpolating control trajectories with B-splines and Akima splines, MTP ensures smooth and globally diverse control candidates. We further propose a simple $\beta$-mixing strategy that blends local exploitative and global exploratory samples within the modified Cross-Entropy Method (CEM) update, balancing control refinement and exploration. Theoretically, we show that MTP achieves asymptotic path coverage and maximum entropy in the control trajectory space in the limit of infinite tensor depth and width. Our implementation is fully vectorized using JAX and compatible with MuJoCo XLA, supporting \emph{Just-in-time} (JIT) compilation and batched rollouts for real-time control with online domain randomization. Through experiments on various challenging robotic tasks, ranging from dexterous in-hand manipulation to humanoid locomotion, we demonstrate that MTP outperforms standard MPC and evolutionary strategy baselines in task success and control robustness. Design and sensitivity ablations confirm the effectiveness of MTP tensor sampling structure, spline interpolation choices, and mixing strategy. Altogether, MTP offers a scalable framework for robust exploration in model-based planning and control.
This paper presents a robust adaptive learning Model Predictive Control (MPC) framework for linear systems with parametric uncertainties and additive disturbances performing iterative tasks. The approach iteratively refines the parameter estimates using set membership estimation. Performance enhancement over iterations is achieved by learning the terminal cost from data. Safety is enforced using a terminal set, which is also learned iteratively. The proposed method guarantees recursive feasibility, constraint satisfaction, and a robust bound on the closed-loop cost. Numerical simulations on a mass-spring-damper system demonstrate improved computational efficiency and control performance compared to an existing robust adaptive MPC approach.
We study residual dynamics learning for differentially flat systems, where a nominal model is augmented with a learned correction term from data. A key challenge is that generic residual parameterizations may destroy flatness, limiting the applicability of flatness-based planning and control methods. To address this, we propose a framework for learning flatness-preserving residual dynamics in systems whose nominal model admits a pure-feedback form. We show that residuals with a lower-triangular structure preserve both the flatness of the system and the original flat outputs. Moreover, we provide a constructive procedure to recover the flatness diffeomorphism of the augmented system from that of the nominal model. We then introduce a learning algorithm that fits such residuals from trajectory data using smooth function approximators. Our approach is validated in simulation on a 2D quadrotor subject to unmodeled aerodynamic effects. We demonstrate that the resulting learned flat model enables tracking performance comparable to nonlinear model predictive control ($5\times$ lower tracking error than the nominal flat model) while also achieving over a $20\times$ speedup in computation.
We study the control of stochastic discrete-time linear multi-agent systems (MAS) subject to additive stochastic noise and collaborative signal temporal logic (STL) specifications to be satisfied with a desired probability. Given available disturbance datasets, we leverage conformal prediction (CP) to address the underlying chance-constrained multi-agent STL synthesis problem in a distribution-free manner. By introducing nonconformity scores as functions of prediction regions (PRs) of error trajectories, we develop an iterative PR-scaling and disturbance-feedback synthesis approach to bound training error trajectory samples. These bounds are then calibrated using a separate dataset, providing probabilistic guarantees via CP. Subsequently, we relax the underlying stochastic optimal control problem by tightening the robustness functions of collaborative tasks based on their Lipschitz constants and the computed error bounds. To address scalability, we exploit the compositional structure of the multi-agent STL formula and propose a model-predictive-control-like algorithm, where agent-level problems are solved in a distributed fashion. Lastly, we showcase the benefits of the proposed method in comparison with [1] via an illustrative example.
Safety assurance in multi-agent systems with coupled dynamics is a fundamental yet challenging problem, especially when agents exhibit uncertain and state-dependent behaviors. Classical robust control often assumes worst-case disturbances, leading to overly conservative actions. In this work, we propose a learning-based framework that combines conformal prediction with model predictive control (MPC) to ensure probabilistic safety under action-level uncertainty. Unlike prior approaches that predict future states, we directly model the control action of the uncontrollable agent as a stochastic function of the joint state, trained via neural networks and calibrated using conformal prediction. This enables us to construct dynamic, probabilistically guaranteed reachable tubes for the uncontrollable agent. These tubes are then embedded into an MPC formulation to synthesize control actions for the controllable agent that ensure safe interactions over a finite planning horizon. We provide formal stepwise and cumulative safety guarantees, and demonstrate the effectiveness of our approach through a pedestrian-vehicle interaction scenario. Compared to baseline methods, our framework achieves higher safety rates while maintaining high performance in terms of speed and responsiveness.
High-dimensional nonlinear systems pose considerable challenges for modeling and control across many domains, from fluid mechanics to advanced robotics. Such systems are typically approximated with reduced order models, which often rely on orthogonal projections, a simplification that may lead to large prediction errors. In this work, we derive optimality of fiber-aligned projections onto spectral submanifolds, preserving the nonlinear geometric structure and minimizing long-term prediction error. We propose a computationally tractable procedure to approximate these projections from data, and show how the effect of control can be incorporated. For a 180-dimensional robotic system, we demonstrate that our reduced-order models outperform previous state-of-the-art approaches by up to fivefold in trajectory tracking accuracy under model predictive control.
The reinforcement learning (RL) and model predictive control (MPC) communities have developed vast ecosystems of theoretical approaches and computational tools for solving optimal control problems. Given their conceptual similarities but differing strengths, there has been increasing interest in synergizing RL and MPC. However, existing approaches tend to be limited for various reasons, including computational cost of MPC in an RL algorithm and software hurdles towards seamless integration of MPC and RL tools. These challenges often result in the use of "simple" MPC schemes or RL algorithms, neglecting the state-of-the-art in both areas. This paper presents MPCritic, a machine learning-friendly architecture that interfaces seamlessly with MPC tools. MPCritic utilizes the loss landscape defined by a parameterized MPC problem, focusing on "soft" optimization over batched training steps; thereby updating the MPC parameters while avoiding costly minimization and parametric sensitivities. Since the MPC structure is preserved during training, an MPC agent can be readily used for online deployment, where robust constraint satisfaction is paramount. We demonstrate the versatility of MPCritic, in terms of MPC architectures and RL algorithms that it can accommodate, on classic control benchmarks.
Data-enabled predictive control (DeePC) has emerged as a powerful technique to control complex systems without the need for extensive modeling efforts. However, relying solely on offline collected data trajectories to represent the system dynamics introduces certain drawbacks. Therefore, we present a novel semi-data-driven model predictive control (SD-MPC) framework that combines (limited) model information with DeePC to address a range of these drawbacks, including sensitivity to noisy data, lack of robustness, and a high computational burden. In this work we focus on the performance of DeePC in operating regimes not captured by the offline collected data trajectories and demonstrate how incorporating an underlying parametric model can counteract this issue. SD-MPC exhibits equivalent closed-loop performance as DeePC for deterministic linear time-invariant systems. Simulations demonstrate the general control performance of the proposed SD-MPC for both a linear time-invariant system and a nonlinear system modeled as a linear parameter-varying system. These results provide numerical evidence of the enhanced robustness of SD-MPC over classical DeePC.
This paper proposes tackling safety-critical stochastic Reinforcement Learning (RL) tasks with a samplebased, model-based approach. At the core of the method lies a Model Predictive Control (MPC) scheme that acts as function approximation, providing a model-based predictive control policy. To ensure safety, a probabilistic Control Barrier Function (CBF) is integrated into the MPC controller. A sample-based approach with guarantees is employed to approximate the effects of stochasticies in the optimal control formulation and to guarantee the probabilistic CBF condition. A learnable terminal cost formulation is included in the MPC objective to counterbalance the additional computational burden due to sampling. An RL algorithm is deployed to learn both the terminal cost and the CBF constraint. Results from our numerical experiment on a constrained LTI problem corroborate the effectiveness of the proposed methodology in reducing computation time while preserving control performance and safety.