The performance of frequency regulating units for automatic generation control (AGC) of power systems depends on their ability to track the AGC signal accurately. In addition, representative models and advanced analysis and analytics can yield forecasts of the AGC signal that aids in controller design. In this paper, time-series analyses are conducted on an AGC signal, specifically the PJM Reg-D, and using the results, a statistical model is derived that fairly accurately captures its second moments and saturated nature, as well as a time-series-based predictive model to provide forecasts. As an application, the predictive model is used in a model predictive control framework to ensure optimal tracking performance of a down ramp-limited distributed energy resource coordination scheme. The results provide valuable insight into the properties of the AGC signal and indicate the effectiveness of these models in replicating its behavior.
This paper presents a novel trajectory tracker for autonomous quadrotor navigation in dynamic and complex environments. The proposed framework integrates a distributional Reinforcement Learning (RL) estimator for unknown aerodynamic effects into a Stochastic Model Predictive Controller (SMPC) for trajectory tracking. Aerodynamic effects derived from drag forces and moment variations are difficult to model directly and accurately. Most current quadrotor tracking systems therefore treat them as simple `disturbances' in conventional control approaches. We propose Quantile-approximation-based Distributional Reinforced-disturbance-estimator, an aerodynamic disturbance estimator, to accurately identify disturbances, i.e., uncertainties between the true and estimated values of aerodynamic effects. Simplified Affine Disturbance Feedback is employed for control parameterization to guarantee convexity, which we then integrate with a SMPC to achieve sufficient and non-conservative control signals. We demonstrate our system to improve the cumulative tracking errors by at least 66% with unknown and diverse aerodynamic forces compared with recent state-of-the-art. Concerning traditional Reinforcement Learning's non-interpretability, we provide convergence and stability guarantees of Distributional RL and SMPC, respectively, with non-zero mean disturbances.
Bridging model-based safety and model-free reinforcement learning (RL) for dynamic robots is appealing since model-based methods are able to provide formal safety guarantees, while RL-based methods are able to exploit the robot agility by learning from the full-order system dynamics. However, current approaches to tackle this problem are mostly restricted to simple systems. In this paper, we propose a new method to combine model-based safety with model-free reinforcement learning by explicitly finding a low-dimensional model of the system controlled by a RL policy and applying stability and safety guarantees on that simple model. We use a complex bipedal robot Cassie, which is a high dimensional nonlinear system with hybrid dynamics and underactuation, and its RL-based walking controller as an example. We show that a low-dimensional dynamical model is sufficient to capture the dynamics of the closed-loop system. We demonstrate that this model is linear, asymptotically stable, and is decoupled across control input in all dimensions. We further exemplify that such linearity exists even when using different RL control policies. Such results point out an interesting direction to understand the relationship between RL and optimal control: whether RL tends to linearize the nonlinear system during training in some cases. Furthermore, we illustrate that the found linear model is able to provide guarantees by safety-critical optimal control framework, e.g., Model Predictive Control with Control Barrier Functions, on an example of autonomous navigation using Cassie while taking advantage of the agility provided by the RL-based controller.
This paper investigates options to complement a diesel engine airpath feedback controller with a feedforward. The control objective is to track the intake manifold pressure and exhaust gas recirculation (EGR) rate targets by manipulating the EGR valve and variable geometry turbine (VGT) while satisfying state and input constraints. The feedback controller is based on rate-based Model Predictive Control (MPC) that provides integral action for tracking. Two options for the feedforward are considered one based on a look-up table that specifies the feedforward as a function of engine speed and fuel injection rate, and another one based on a (non-rate-based) MPC that generates dynamic feedforward trajectories. The controllers are designed and verified using a high-fidelity engine model in GT-Power and exploit a low-order rate-based linear parameter-varying (LPV) model for prediction which is identified from transient response data generated by the GT-Power model. It is shown that the combination of feedforward and feedback MPC has the potential to improve the performance and robustness of the control design. In particular, the feedback MPC without feedforward can lose stability at low engine speeds, while MPC-based feedforward results in the best transient response. Mechanisms by which feedforward is able to assist in stabilization and improve performance are discussed.
Many chemical processes exhibit diverse timescale dynamics with a strong coupling between timescale sensitive variables. Model predictive control with a non-uniformly spaced optimisation horizon is an effective approach to multi-timescale control and offers opportunities for reduced computational complexity. In such an approach the fast, moderate and slow dynamics can be included in the optimisation problem by implementing smaller time intervals earlier in the prediction horizon and increasingly larger intervals towards the end of the prediction. In this paper, a reference-flexible condition is developed based on the contraction theory to provide a stability guarantee for a nonlinear system under non-uniform prediction horizons.
Integrated simulation models are emerging as an alternative for analyzing large-scale interdependent infrastructure networks due to their modeling advantages over traditional interdependency models. This paper presents an open-source integrated simulation package for the asset-level analysis of interdependent infrastructure systems. The simulation platform, named 'InfraRisk' and developed in Python, can simulate disaster-induced infrastructure failures and subsequent post-disaster restoration in interconnected water-, power-, and road networks. InfraRisk consists of an infrastructure module, a hazard module, a recovery module, a simulation module, and a resilience quantification module. The infrastructure module integrates existing infrastructure network packages (wntr for water networks, pandapower for power systems, and a static traffic assignment model for transportation networks) through an interface that facilitates the network-level simulation of interdependent failures. The hazard module generates infrastructure component failure sequences based on various disaster characteristics. The recovery module determines repair sequences and assigns repair crews based on predefined heuristics-based recovery strategies or model predictive control (MPC) based optimization. Based on the schedule, the simulation module implements the network-wide simulation of the consequences of the disaster impacts and the recovery actions. The resilience quantification module offers system-level and consumer-level metrics to quantify both the risks and resilience of the integrated infrastructure networks against disaster events. InfraRisk provides a virtual platform for decision-makers to experiment and develop region-specific pre-disaster and post-disaster policies to enhance the overall resilience of interdependent urban infrastructure networks.
Nonlinear model predictive control (NMPC) has gained widespread use in many applications. Its formulation traditionally involves repetitively solving a nonlinear constrained optimization problem online. In this paper, we investigate NMPC through the lens of Bayesian estimation and highlight that the Monte Carlo sampling method can offer a favorable way to implement NMPC. We develop a constraint-aware particle filtering/smoothing method and exploit it to implement NMPC. The new sampling-based NMPC algorithm can be executed easily and efficiently even for complex nonlinear systems, while potentially mitigating the issues of computational complexity and local minima faced by numerical optimization in conventional studies. The effectiveness of the proposed algorithm is evaluated through a simulation study.
We propose a Model Predictive Control (MPC) method for collision-free navigation that uses amortized variational inference to approximate the distribution of optimal control sequences by training a normalizing flow conditioned on the start, goal and environment. This representation allows us to learn a distribution that accounts for both the dynamics of the robot and complex obstacle geometries. We can then sample from this distribution to produce control sequences which are likely to be both goal-directed and collision-free as part of our proposed FlowMPPI sampling-based MPC method. However, when deploying this method, the robot may encounter an out-of-distribution (OOD) environment, i.e. one which is radically different from those used in training. In such cases, the learned flow cannot be trusted to produce low-cost control sequences. To generalize our method to OOD environments we also present an approach that performs projection on the representation of the environment as part of the MPC process. This projection changes the environment representation to be more in-distribution while also optimizing trajectory quality in the true environment. Our simulation results on a 2D double-integrator and a 3D 12DoF underactuated quadrotor suggest that FlowMPPI with projection outperforms state-of-the-art MPC baselines on both in-distribution and OOD environments, including OOD environments generated from real-world data.
Model predictive control (MPC) has become the most widely used advanced control method in process industry. In many cases, forecasts of the disturbances are available, e.g., predicted renewable power generation based on weather forecast. While the predictions of disturbances may not be accurate, utilizing the information can significantly improve the control performance in response to the disturbances. By exploiting process and disturbance models, future system behaviour can be predicted and used to optimise control actions via minimisation of an economical cost function which incorporates these predictions. However, stability guarantee of the resulting closed-loop system is often difficult in this approach when the processes are nonlinear. Proposed in the following article is a contraction-constrained predictive controller which optimises process economy whilst ensuring stabilisation to operating targets subject to disturbance measurements and forecasts.
In this work, we study economic model predictive control (MPC) in situations where the optimal operating behavior is periodic. In such a setting, the performance of a standard economic MPC scheme without terminal conditions can generally be far from optimal even with arbitrarily long prediction horizons. Whereas there are modified economic MPC schemes that guarantee optimal performance, all of them are based on prior knowledge of the optimal period length or of the optimal periodic orbit itself. In contrast to these approaches, we propose to achieve optimality by multiplying the stage cost by a linear discount factor. This modification is not only easy to implement but also independent of any system- or cost-specific properties, making the scheme robust against online changes therein. Under standard dissipativity and controllability assumptions, we can prove that the resulting linearly discounted economic MPC without terminal conditions achieves optimal asymptotic average performance up to an error that vanishes with growing prediction horizons. Moreover, we can guarantee practical asymptotic stability of the optimal periodic orbit under the additional technical assumption that dissipativity holds with a continuous storage function. We complement these qualitative guarantees with a quantitative analysis of the transient and asymptotic average performance of the linearly discounted MPC scheme in a numerical simulation study.
Modeling and manipulating elasto-plastic objects are essential capabilities for robots to perform complex industrial and household interaction tasks (e.g., stuffing dumplings, rolling sushi, and making pottery). However, due to the high degree of freedom of elasto-plastic objects, significant challenges exist in virtually every aspect of the robotic manipulation pipeline, e.g., representing the states, modeling the dynamics, and synthesizing the control signals. We propose to tackle these challenges by employing a particle-based representation for elasto-plastic objects in a model-based planning framework. Our system, RoboCraft, only assumes access to raw RGBD visual observations. It transforms the sensing data into particles and learns a particle-based dynamics model using graph neural networks (GNNs) to capture the structure of the underlying system. The learned model can then be coupled with model-predictive control (MPC) algorithms to plan the robot's behavior. We show through experiments that with just 10 minutes of real-world robotic interaction data, our robot can learn a dynamics model that can be used to synthesize control signals to deform elasto-plastic objects into various target shapes, including shapes that the robot has never encountered before. We perform systematic evaluations in both simulation and the real world to demonstrate the robot's manipulation capabilities and ability to generalize to a more complex action space, different tool shapes, and a mixture of motion modes. We also conduct comparisons between RoboCraft and untrained human subjects controlling the gripper to manipulate deformable objects in both simulation and the real world. Our learned model-based planning framework is comparable to and sometimes better than human subjects on the tested tasks.
In automotive power electronics, distortion and electromagnetic interference (EMI) generated by the switching action of power semiconductors can be a significant challenge for the design of a compact, lightweight vehicle. As semiconductor switching frequencies increase, e.g., through the introduction of new materials, such as gallium nitride and silicon carbide, this problem becomes more severe. We present a control scheme for an automotive dc-to-dc converter that reduces the EMI generated by shaping switching distortion predictively at the run time. The multistep model-predictive control scheme chooses the subsequent switching state that optimizes the output spectrum according to predefined criteria. To achieve real-time operation, it evaluates the possible switching state candidates for the next modulation step without explicitly solving a single Fourier transform. In addition, the switching rate and voltage ripple are controlled in a single unified control law. We present and experimentally validate that the control scheme can indeed run at real time already with currently available mid-range hardware. The results demonstrate that the largest spectral peak of the switching distortion can be decreased by 48 dB compared to conventional pulse-width modulation. Furthermore, spectral gaps can be implemented in the output distortion and altered in real-time, allowing certain frequency bands -- e.g., bands used by other sensitive electronics such as sensors, communication busses, or tuners -- to be kept free of EMI from the converter's switching actions.
The increasing access to non-expensive sensors, computing power, and more accurate forecasting of storm events provides unique opportunities to shift flood management practices from static approaches to an optimization-based real-time control (RTC) of urban drainage systems. Recent studies have addressed a plethora of strategies for flood control in stormwater reservoirs; however, advanced control theoretic techniques are not yet fully investigated and applied to these systems. In addition, there is an absence of a coupled integrated control model for systems composed of watersheds, reservoirs, and channels for flood mitigation. To this end, we develop a novel state-space model of hydrologic and hydrodynamic processes in reservoirs and one-dimensional channels. The model is tested under different types of reservoir control strategies based on real-time measurements (reactive control), and based on predictions of the future behavior of the system (predictive control) using rainfall forecastings. We apply the modeling approach in a system composed by a single watershed, reservoir, and a channel connected in series, respectively, for the San Antonio observed rainfall data. Results indicate that for flood mitigation, the predictive control strategy outperforms the reactive controls not only when applied for synthetic design storm events, but also for a continuous simulation. Moreover, the predictive control strategy requires smaller valve operations, while still guaranteeing efficient hydrological performance. From the results, we recommend the use of the model predictive control strategy to control stormwater systems due to the ability to handle different objective functions, which can be altered according to rainfall forecasting and shift the reservoir operation from flood-based control to strategies focused on increasing detention times, depending on the forecasting.
Water scarcity is an urgent issue to be resolved and improving irrigation water-use efficiency through closed-loop control is essential. The complex agro-hydrological system dynamics, however, often pose challenges in closed-loop control applications. In this work, we propose a two-layer neural network (NN) framework to approximate the dynamics of the agro-hydrological system. To minimize the prediction error, a linear bias correction is added to the proposed model. The model is employed by a model predictive controller with zone tracking (ZMPC), which aims to keep the root zone soil moisture in the target zone while minimizing the total amount of irrigation. The performance of the proposed approximation model framework is shown to be better compared to a benchmark long-short-term-memory (LSTM) model for both open-loop and closed-loop applications. Significant computational cost reduction of the ZMPC is achieved with the proposed framework. To handle the tracking offset caused by the plant-model-mismatch of the proposed NN framework, a shrinking target zone is proposed for the ZMPC. Different hyper-parameters of the shrinking zone in the presence of noise and weather disturbances are investigated, of which the control performance is compared to a ZMPC with a time-invariant target zone.
Tube-enhanced multi-stage nonlinear model predictive control is a robust control scheme that can handle a wide range of uncertainties with reduced conservatism and manageable computational complexity. In this paper, we elaborate on the flexibility of the approach from an application point of view. We discuss the path to making design decisions to implement the novel scheme systematically. We illustrate the critical steps in the design and implementation of the scheme for an industrial example.
This paper studies the problem of risk-averse receding horizon motion planning for agents with uncertain dynamics, in the presence of stochastic, dynamic obstacles. We propose a model predictive control (MPC) scheme that formulates the obstacle avoidance constraint using coherent risk measures. To handle disturbances, or process noise, in the state dynamics, the state constraints are tightened in a risk-aware manner to provide a disturbance feedback policy. We also propose a waypoint following algorithm that uses the proposed MPC scheme for discrete distributions and prove its risk-sensitive recursive feasibility while guaranteeing finite-time task completion. We further investigate some commonly used coherent risk metrics, namely, conditional value-at-risk (CVaR), entropic value-at-risk (EVaR), and g-entropic risk measures, and propose a tractable incorporation within MPC. We illustrate our framework via simulation studies.
There is a growing debate on whether the future of feedback control systems will be dominated by data-driven or model-driven approaches. Each of these two approaches has their own complimentary set of advantages and disadvantages, however, only limited attempts have, so far, been developed to bridge the gap between them. To address this issue, this paper introduces a method to bound the worst-case error between feedback control policies based upon model predictive control (MPC) and neural networks (NNs). This result is leveraged into an approach to automatically synthesize MPC policies minimising the worst-case error with respect to a NN. Numerical examples highlight the application of the bounds, with the goal of the paper being to encourage a more quantitative understanding of the relationship between data-driven and model-driven control.
Combining efficient and safe control for safety-critical systems is challenging. Robust methods may be overly conservative, whereas probabilistic controllers require a trade-off between efficiency and safety. In this work, we propose a safety algorithm that is compatible with any stochastic Model Predictive Control method for linear systems with additive uncertainty and polytopic constraints. This safety algorithm allows to use the optimistic control inputs of stochastic Model Predictive Control as long as a safe backup planner can ensure safety with respect to satisfying hard constraints subject to bounded uncertainty. Besides ensuring safe behavior, the proposed stochastic Model Predictive Control algorithm guarantees recursive feasibility and input-to-state stability of the system origin. The benefits of the safe stochastic Model Predictive Control algorithm are demonstrated in a numerical simulation, highlighting the advantages compared to purely robust or stochastic predictive controllers.
In this work, we combine {Model Predictive Control} (MPC) and Control Barrier Function (CBF) design {methods} to create a hierarchical control law for simultaneous lane-keeping (LK) and obstacle avoidance (OA): at the low level, MPC performs LK via trajectory tracking during nominal operation; and at the high level, different CBF-based safety filters that ensure both LK and OA are designed and compared across some practical scenarios. In particular, we show that Exponential Safety (ESf) and Prescribed-Time Safety (PTSf) filters, which override the MPC control when necessary, result in feasible Quadratic Programs when safety is prioritized appropriately. We additionally investigate control designs subject to input constraints by using Input-Constrained-CBFs. Finally, we compare the performance of combinations of ESf, PTSf, and their input-constrained counterparts with respect to the LK and OA goals in two simulation studies for early- and late-detected obstacle scenarios.
This work explores the nature of augmented importance sampling in safety-constrained model predictive control problems. When operating in a constrained environment, sampling based model predictive control and motion planning typically utilizes penalty functions or expensive optimization based control barrier algorithms to maintain feasibility of forward sampling. In contrast the presented algorithm utilizes discrete embedded barrier states in augmented importance sampling to apply feedback with respect to a nominal state when sampling. We will demonstrate that this approach of safety of discrete embedded barrier states in augmented importance sampling is more sample efficient by metric of collision free trajectories, is computationally feasible to perform per sample, and results in better safety performance on a cluttered navigation task with extreme un-modeled disturbances. In addition, we will utilize the theoretical properties of augmented importance sampling and safety control to derive a new bound on the free energy of the system.