While MPC enables nonlinear feedback control by solving an optimal control problem at each timestep, the computational burden tends to be significantly large, making it difficult to optimize a policy within the control period. To address this issue, one possible approach is to utilize terminal value learning to reduce computational costs. However, the learned value cannot be used for other tasks in situations where the task dynamically changes in the original MPC setup. In this study, we develop an MPC framework with goal-conditioned terminal value learning to achieve multitask policy optimization while reducing computational time. Furthermore, by using a hierarchical control structure that allows the upper-level trajectory planner to output appropriate goal-conditioned trajectories, we demonstrate that a robot model is able to generate diverse motions. We evaluate the proposed method on a bipedal inverted pendulum robot model and confirm that combining goal-conditioned terminal value learning with an upper-level trajectory planner enables real-time control; thus, the robot successfully tracks a target trajectory on sloped terrain.
Integrated energy systems (IESs) are complex systems consisting of diverse operating units spanning multiple domains. To address its operational challenges, we propose a physics-informed hybrid time-series neural network (NN) surrogate to predict the dynamic performance of IESs across multiple time scales. This neural network-based modeling approach develops time-series multi-layer perceptrons (MLPs) for the operating units and integrates them with prior process knowledge about system structure and fundamental dynamics. This integration forms three hybrid NNs (long-term, slow, and fast MLPs) that predict the entire system dynamics across multiple time scales. Leveraging these MLPs, we design an NN-based scheduler and an NN-based economic model predictive control (NEMPC) framework to meet global operational requirements: rapid electrical power responsiveness to operators requests, adequate cooling supply to customers, and increased system profitability, while addressing the dynamic time-scale multiplicity present in IESs. The proposed day-ahead scheduler is formulated using the ReLU network-based MLP, which effectively represents IES performance under a broad range of conditions from a long-term perspective. The scheduler is then exactly recast into a mixed-integer linear programming problem for efficient evaluation. The real-time NEMPC, based on slow and fast MLPs, comprises two sequential distributed control agents: a slow NEMPC for the cooling-dominant subsystem with slower transient responses and a fast NEMPC for the power-dominant subsystem with faster responses. Extensive simulations demonstrate that the developed scheduler and NEMPC schemes outperform their respective benchmark scheduler and controller by about 25% and 40%. Together, they enhance overall system performance by over 70% compared to benchmark approaches.
Model-based reinforcement learning has shown promise for improving sample efficiency and decision-making in complex environments. However, existing methods face challenges in training stability, robustness to noise, and computational efficiency. In this paper, we propose Bisimulation Metric for Model Predictive Control (BS-MPC), a novel approach that incorporates bisimulation metric loss in its objective function to directly optimize the encoder. This time-step-wise direct optimization enables the learned encoder to extract intrinsic information from the original state space while discarding irrelevant details and preventing the gradients and errors from diverging. BS-MPC improves training stability, robustness against input noise, and computational efficiency by reducing training time. We evaluate BS-MPC on both continuous control and image-based tasks from the DeepMind Control Suite, demonstrating superior performance and robustness compared to state-of-the-art baseline methods.
Navigating complex environments requires Unmanned Aerial Vehicles (UAVs) and autonomous systems to perform trajectory tracking and obstacle avoidance in real-time. While many control strategies have effectively utilized linear approximations, addressing the non-linear dynamics of UAV, especially in obstacle-dense environments, remains a key challenge that requires further research. This paper introduces a Non-linear Model Predictive Control (NMPC) framework for the DJI Matrice 100, addressing these challenges by using a dynamic model and B-spline interpolation for smooth reference trajectories, ensuring minimal deviation while respecting safety constraints. The framework supports various trajectory types and employs a penalty-based cost function for control accuracy in tight maneuvers. The framework utilizes CasADi for efficient real-time optimization, enabling the UAV to maintain robust operation even under tight computational constraints. Simulation and real-world indoor and outdoor experiments demonstrated the NMPC ability to adapt to disturbances, resulting in smooth, collision-free navigation.
The operation of an autonomous underwater vehicle (AUV) faces challenges in following predetermined waypoints due to coupled motions under environmental disturbances. To address this, a 3D path following guidance and control system is developed in this work based on the line-of-sight (LOS) guidance method. Conventionally, the 3D path following problem is transformed into heading and depth control problems, assuming that the motion of the vehicle is decoupled in horizontal and depth coordinates. The proposed control system design avoids this simplifying assumption by transforming the problem into a 3D position and orientation tracking problem. This design is achieved by computing a 2D horizontal coordinate based on the desired heading and then computing a corresponding LOS depth coordinate. A model predictive controller (MPC) is then implemented using the 3D LOS coordinate and the computed orientation vector. The MPC obtains a robust control by solving a minimax optimisation problem considering the effects of unknown ocean disturbances. The effectiveness of the proposed guidance and control system is demonstrated through the simulation of a prototype AUV system. Numerical results show that the AUV can follow predetermined waypoints in the presence of time-varying disturbances, and the system is steered at a constant surge speed that is proportional to the radius of the circle of acceptance used to implement the guidance system.
This paper investigates the existence of a separation principle between model identification and control design in the context of model predictive control. First, we elucidate that the separation principle holds asymptotically in the number of data in a Fisherian setting, and universally in a Bayesian setting. Then, by formulating model predictive control within a Gaussian regression framework, we describe how the Bayesian separation principle can be used to derive explicit, uncertainty-aware expressions for the control cost and optimal input sequence, thereby bridging direct and indirect data-driven approaches.
Greenhouse climate control is concerned with maximizing performance in terms of crop yield and resource efficiency. One promising approach is model predictive control (MPC), which leverages a model of the system to optimize the control inputs, while enforcing physical constraints. However, prediction models for greenhouse systems are inherently inaccurate due to the complexity of the real system and the uncertainty in predicted weather profiles. For model-based control approaches such as MPC, this can degrade performance and lead to constraint violations. Existing approaches address uncertainty in the prediction model with robust or stochastic MPC methodology; however, these necessarily reduce crop yield due to conservatism and often bear higher computational loads. In contrast, learning-based control approaches, such as reinforcement learning (RL), can handle uncertainty naturally by leveraging data to improve performance. This work proposes an MPC-based RL control framework to optimize the climate control performance in the presence of prediction uncertainty. The approach employs a parametrized MPC scheme that learns directly from data, in an online fashion, the parametrization of the constraints, prediction model, and optimization cost that minimizes constraint violations and maximizes climate control performance. Simulations show that the approach can learn an MPC controller that significantly outperforms the current state-of-the-art in terms of constraint violations and efficient crop growth.
Current research on robust trajectory planning for autonomous agents aims to mitigate uncertainties arising from disturbances and modeling errors while ensuring guaranteed safety. Existing methods primarily utilize stochastic optimal control techniques with chance constraints to maintain a minimum distance among agents with a guaranteed probability. However, these approaches face challenges, such as the use of simplifying assumptions that result in linear system models or Gaussian disturbances, which limit their practicality in complex realistic scenarios. To address these limitations, this work introduces a novel probabilistically robust distributed controller enabling autonomous agents to plan safe trajectories, even under non-Gaussian uncertainty and nonlinear systems. Leveraging exact uncertainty propagation techniques based on mixed-trigonometric-polynomial moment propagation, this method transforms non-Gaussian chance constraints into deterministic ones, seamlessly integrating them into a distributed model predictive control framework solvable with standard optimization tools. Simulation results demonstrate the effectiveness of this technique, highlighting its ability to consistently handle various types of uncertainty, ensuring robust and accurate path planning in complex scenarios.
This letter presents a Koopman-theoretic lifted linear parameter-varying (LPV) system with countably infinite dimensions to model the nonlinear dynamics of a quadrotor on SE(3) for facilitating control design. The LPV system evolves in time in the space of the observables, called the lifted space. A primary challenge in utilizing the Koopman-based linearization is identifying a set of observables that can adequately span the lifted space, with the majority of the current methods using data to learn these observables. In this study, we analytically derive the observables for the quadrotor dynamics on SE(3) to formulate the lifted LPV system. The lifted LPV system has a countably infinite dimension which is then truncated for practical control design. The truncation is analytically justified by showing vanishing residual property in a bounded trajectory regime. The LPV system is then approximated as a linear time-invariant (LTI) system with a set of virtual control inputs. The controllability of the lifted LTI system is translatable to the true quadrotor system on SE(3). A linear model-predictive control (LMPC) scheme is formulated and implemented in numerical simulations employing this LTI framework for various tracking problems, with attention given to the potential for real-time implementation.
Model Predictive Control (MPC) is a common tool for the control of nonlinear, real-world systems, such as legged robots. However, solving MPC quickly enough to enable its use in real-time is often challenging. One common solution is given by real-time iterations, which does not solve the MPC problem to convergence, but rather close enough to give an approximate solution. In this paper, we extend this idea to a bilevel control framework where a "high-level" optimization program modifies a controller parameter of a "low-level" MPC problem which generates the control inputs and desired state trajectory. We propose an algorithm to iterate on this bilevel program in real-time and provide conditions for its convergence and improvements in stability. We then demonstrate the efficacy of this algorithm by applying it to a quadrupedal robot where the high-level problem optimizes a contact schedule in real-time. We show through simulation that the algorithm can yield improvements in disturbance rejection and optimality, while creating qualitatively new gaits.
Constraint admissible positively invariant (CAPI) sets play a pivotal role in ensuring safety in control and planning applications, such as the recursive feasibility guarantee of explicit reference governor and model predictive control. However, existing methods for finding CAPI sets for nonlinear systems are often limited to single equilibria or specific system dynamics. This limitation underscores the necessity for a method to construct a CAPI set for general reference tracking control and a broader range of systems. In this work, we leverage recent advancements in learning-based methods to derive Lyapunov functions, particularly focusing on those with piecewise-affine activation functions. Previous attempts to find an invariant set with the piecewise-affine neural Lyapunov function have focused on the estimation of the region of attraction with mixed integer programs. We propose a methodology to determine the maximal CAPI set for any reference with the neural Lyapunov function by transforming the problem into multiple linear programs. Additionally, to enhance applicability in real-time control scenarios, we introduce a learning-based approach to train the estimator, which infers the CAPI set from a given reference. The proposed approach is validated with multiple simulations to show that it can generate a valid CAPI set with the given neural Lyapunov functions for any reference. We also employ the proposed CAPI set estimation method in the explicit reference governor and demonstrate its effectiveness for constrained control.
Networked control systems enable real-time control and coordination of distributed systems, leveraging the low latency, high reliability, and massive connectivity offered by 5G and future 6G networks. Applications include autonomous vehicles, robotics, industrial automation, and smart grids. Despite networked control algorithms admitting nominal stability guarantees even in the presence of delays and packet dropouts, their practical performance still heavily depends on the specific characteristics and conditions of the underlying network. To achieve the desired performance while efficiently using communication resources, co-design of control and communication is pivotal. Although periodic schemes, where communication instances are fixed, can provide reliable control performance, unnecessary transmissions, when updates are not needed, result in inefficient usage of network resources. In this paper, we investigate the potential for co-design of model predictive control and network communication. To this end, we design and implement an event-triggered nonlinear model predictive controller for stabilizing a Furuta pendulum communicating over a tailored open radio access network 6G research platform. We analyze the control performance as well as network utilization under varying channel conditions and event-triggering criteria. Additionally, we analyze the network-induced delay pattern and its interaction with the event-triggered controller. Our results show that the event-triggered control scheme achieves similar performance to periodic control with reduced communication demand.
Recent work in imitation learning has shown that having an expert controller that is both suitably smooth and stable enables stronger guarantees on the performance of the learned controller. However, constructing such smoothed expert controllers for arbitrary systems remains challenging, especially in the presence of input and state constraints. As our primary contribution, we show how such a smoothed expert can be designed for a general class of systems using a log-barrier-based relaxation of a standard Model Predictive Control (MPC) optimization problem. Improving upon our previous work, we show that barrier MPC achieves theoretically optimal error-to-smoothness tradeoff along some direction. At the core of this theoretical guarantee on smoothness is an improved lower bound we prove on the optimality gap of the analytic center associated with a convex Lipschitz function, which we believe could be of independent interest. We validate our theoretical findings via experiments, demonstrating the merits of our smoothing approach over randomized smoothing.
We introduce a Model Predictive Control (MPC) framework for training deep neural networks, systematically unifying the Back-Propagation (BP) and Forward-Forward (FF) algorithms. At the same time, it gives rise to a range of intermediate training algorithms with varying look-forward horizons, leading to a performance-efficiency trade-off. We perform a precise analysis of this trade-off on a deep linear network, where the qualitative conclusions carry over to general networks. Based on our analysis, we propose a principled method to choose the optimization horizon based on given objectives and model specifications. Numerical results on various models and tasks demonstrate the versatility of our method.
Echo State Networks (ESNs) are recurrent neural networks usually employed for modeling nonlinear dynamic systems with relatively ease of training. By incorporating physical laws into the training of ESNs, Physics-Informed ESNs (PI-ESNs) were proposed initially to model chaotic dynamic systems without external inputs. They require less data for training since Ordinary Differential Equations (ODEs) of the considered system help to regularize the ESN. In this work, the PI-ESN is extended with external inputs to model controllable nonlinear dynamic systems. Additionally, an existing self-adaptive balancing loss method is employed to balance the contributions of the residual regression term and the physics-informed loss term in the total loss function. The experiments with two nonlinear systems modeled by ODEs, the Van der Pol oscillator and the four-tank system, and with one differential-algebraic (DAE) system, an electric submersible pump, revealed that the proposed PI-ESN outperforms the conventional ESN, especially in scenarios with limited data availability, showing that PI-ESNs can regularize an ESN model with external inputs previously trained on just a few datapoints, reducing its overfitting and improving its generalization error (up to 92% relative reduction in the test error). Further experiments demonstrated that the proposed PI-ESN is robust to parametric uncertainties in the ODE equations and that model predictive control using PI-ESN outperforms the one using plain ESN, particularly when training data is scarce.
Real-world (bio)chemical processes often exhibit stochastic dynamics with non-trivial correlations and state-dependent fluctuations. Model predictive control (MPC) often must consider these fluctuations to achieve reliable performance. However, most process models simply add stationary noise terms to a deterministic prediction. This work proposes using conditional normalizing flows as discrete-time models to learn stochastic dynamics. Normalizing flows learn the probability density function (PDF) of the states explicitly, given prior states and control inputs. In addition to standard least squares (LSQ) objectives, this work derives a marginal log-likelihood (MLL) objective based on the explicit PDF and Markov chain simulations. In a reactor study, the normalizing flow MPC reduces the setpoint error in open and closed-loop cases to half that of a nominal controller. Furthermore, the chance constraints lead to fewer constraint violations than the nominal controller. The MLL objective yields slightly more stable results than the LSQ, particularly for small scenario sets.
This paper evaluates and compares the performance of model-free and model-based reinforcement learning for the attitude control of fixed-wing unmanned aerial vehicles using PID as a reference point. The comparison focuses on their ability to handle varying flight dynamics and wind disturbances in a simulated environment. Our results show that the Temporal Difference Model Predictive Control agent outperforms both the PID controller and other model-free reinforcement learning methods in terms of tracking accuracy and robustness over different reference difficulties, particularly in nonlinear flight regimes. Furthermore, we introduce actuation fluctuation as a key metric to assess energy efficiency and actuator wear, and we test two different approaches from the literature: action variation penalty and conditioning for action policy smoothness. We also evaluate all control methods when subject to stochastic turbulence and gusts separately, so as to measure their effects on tracking performance, observe their limitations and outline their implications on the Markov decision process formalism.
Engineering biology requires precise control of biomolecular circuits, and Cybergenetics is the field dedicated to achieving this goal. A significant challenge in developing controllers for cellular functions is designing systems that can effectively manage molecular noise. To address this, there has been increasing effort to develop model-based controllers for stochastic biomolecular systems, where a major difficulty lies in accurately solving the chemical master equation. In this work we develop a framework for optimal and Model Predictive Control of stochastic gene regulatory networks with three key advantageous features: high computational efficiency, the capacity to control the overall probability density function enabling the fine-tuning of the cell population to obtain complex shapes and behaviors (including bimodality and other emergent properties), and the capacity to handle high levels of intrinsic molecular noise. Our method exploits an efficient approximation of the Chemical Master Equation using Partial Integro-Differential Equations, which additionally enables the development of an effective adjoint-based optimization. We illustrate the performance of the methods presented through two relevant studies in Synthetic Biology: shaping bimodal cell populations and tracking moving target distributions via inducible gene regulatory circuits.
Efficient water management in agriculture is important for mitigating the growing freshwater scarcity crisis. Mixed-integer Model Predictive Control (MPC) has emerged as an effective approach for addressing the complex scheduling problems in agricultural irrigation. However, the computational complexity of mixed-integer MPC still poses a significant challenge, particularly in large-scale applications. This study proposes an approach to enhance the computational efficiency of mixed-integer MPC-based irrigation schedulers by employing ReLU surrogate models to describe the soil moisture dynamics of the agricultural field. By leveraging the mixed-integer linear representation of the ReLU operator, the proposed approach transforms the mixed-integer MPC-based scheduler with a quadratic cost function into a mixed-integer quadratic program, which is the simplest class of mixed-integer nonlinear programming problems that can be efficiently solved using global optimization solvers. The effectiveness of this approach is demonstrated through comparative studies conducted on a large-scale agricultural field across two growing seasons, involving other machine learning surrogate models, specifically Long Short-Term Memory (LSTM) networks, and the widely used triggered irrigation scheduling method. The ReLU-based approach significantly reduces solution times -- by up to 99.5\% -- while achieving comparable performance to the LSTM approach in terms of water savings and Irrigation Water Use Efficiency (IWUE). Moreover, the ReLU-based approach maintains enhanced performance in terms of total prescribed irrigation and IWUE compared to the widely-used triggered irrigation scheduling method.
The software architecture behind modern autonomous vehicles (AV) is becoming more complex steadily. Safety verification is now an imminent task prior to the large-scale deployment of such convoluted models. For safety-critical tasks in navigation, it becomes imperative to perform a verification procedure on the trajectories proposed by the planning algorithm prior to deployment. Signal Temporal Logic (STL) constraints can dictate the safety requirements for an AV. A combination of STL constraints is called a specification. A key difference between STL and other logic constraints is that STL allows us to work on continuous signals. We verify the satisfaction of the STL specifications by calculating the robustness value for each signal within the specification. Higher robustness values indicate a safer system. Model Predictive Control (MPC) is one of the most widely used methods to control the navigation of an AV, with an underlying set of state and input constraints. Our research aims to formulate and test an MPC controller, with STL specifications as constraints, that can safely navigate an AV. The primary goal of the cost function is to minimize the control inputs. STL constraints will act as an additional layer of constraints that would change based on the scenario and task on hand. We propose using sTaliro, a MATLAB-based robustness calculator for STL specifications, formulated in a receding horizon control fashion for an AV navigation task. It inputs a simplified AV state space model and a set of STL specifications, for which it constructs a closed-loop controller. We test out our controller for different test cases/scenarios and verify the safe navigation of our AV model.