Coalitional control is concerned with the management of multi-agent systems where cooperation cannot be taken for granted (due to, e.g., market competition, logistics). This paper proposes a model predictive control (MPC) framework aimed at large-scale dynamically-coupled systems whose individual components, possessing a limited model of the system, are controlled independently, pursuing possibly competing objectives. The emergence of cooperating clusters of controllers is contemplated through an autonomous negotiation protocol, based on the characterization as a coalitional game of the benefit derived by a broader feedback and the alignment of the individual objectives. Specific mechanisms for the cooperative benefit redistribution that relax the cognitive requirements of the game are employed to compensate for possible local cost increases due to cooperation. As a result, the structure of the overall MPC feedback can be adapted online to the degree of interaction between different parts of the system, while satisfying the individual interests of the agents. A wide-area control application for the power grid with the objective of minimizing frequency deviations and undesired inter-area power transfers is used as study case.
Ensuring nominal asymptotic stability of the Nonlinear Model Predictive Control controller is not trivial. Stabilizing ingredients such as terminal penalty term and terminal region are crucial in establishing the asymptotic stability. Current work presents alternate approaches namely arbitrary controller based approach and linear quadratic regulator based approach, which provide larger degrees of freedom for enlarging the terminal region as against conservative approaches from the literature. Efficacy of the proposed approaches is demonstrated using benchmark two state continuous stirrer tank reactor system around an unstable operating point. Terminal regions obtained using the arbitrary controller based approach and linear quadratic regulator based approach are approximately 45 and 412 times larger by area measure when compared to the largest terminal region obtained using the approach from the literature. As a result, there is significant reduction in the prediction and control horizon time.
We study learning based controllers as a replacement for model predictive controllers (MPC) for the control of autonomous vehicles. We concentrate for the experiments on the simple yet representative bicycle model. We compare training by supervised learning and by reinforcement learning. We also discuss the neural net architectures so as to obtain small nets with the best performances. This work aims at producing controllers that can both be embedded on real-time platforms and amenable to verification by formal methods techniques.
For economic nonlinear model predictive control and dynamic real-time optimization fast and accurate models are necessary. Consequently, the use of dynamic surrogate models to mimic complex rigorous models is increasingly coming into focus. For dynamic systems, the focus so far had been on identifying a system's behavior surrounding a steady-state operation point. In this contribution, we propose a novel methodology to adaptively sample rigorous dynamic process models to generate a dataset for building dynamic surrogate models. The goal of the developed algorithm is to cover an as large as possible area of the feasible region of the original model. To demonstrate the performance of the presented framework it is applied on a dynamic model of a chlor-alkali electrolysis.
Reinforcement learning (RL) is a technique to learn the control policy for an agent that interacts with a stochastic environment. In any given state, the agent takes some action, and the environment determines the probability distribution over the next state as well as gives the agent some reward. Most RL algorithms typically assume that the environment satisfies Markov assumptions (i.e. the probability distribution over the next state depends only on the current state). In this paper, we propose a model-based RL technique for a system that has non-Markovian dynamics. Such environments are common in many real-world applications such as in human physiology, biological systems, material science, and population dynamics. Model-based RL (MBRL) techniques typically try to simultaneously learn a model of the environment from the data, as well as try to identify an optimal policy for the learned model. We propose a technique where the non-Markovianity of the system is modeled through a fractional dynamical system. We show that we can quantify the difference in the performance of an MBRL algorithm that uses bounded horizon model predictive control from the optimal policy. Finally, we demonstrate our proposed framework on a pharmacokinetic model of human blood glucose dynamics and show that our fractional models can capture distant correlations on real-world datasets.
We consider control from the perspective of competitive analysis. Unlike much prior work on learning-based control, which focuses on minimizing regret against the best controller selected in hindsight from some specific class, we focus on designing an online controller which competes against a clairvoyant offline optimal controller. A natural performance metric in this setting is competitive ratio, which is the ratio between the cost incurred by the online controller and the cost incurred by the offline optimal controller. Using operator-theoretic techniques from robust control, we derive a computationally efficient state-space description of the the controller with optimal competitive ratio in both finite-horizon and infinite-horizon settings. We extend competitive control to nonlinear systems using Model Predictive Control (MPC) and present numerical experiments which show that our competitive controller can significantly outperform standard $H_2$ and $H_{\infty}$ controllers in the MPC setting.
Packetized energy management (PEM) is a demand dispatch scheme that can be used to provide ancillary services such as frequency regulation. In PEM, distributed energy resources (DERs) are granted uninterruptible access to the grid for a pre-specified time interval called the packet length. This results in a down ramp-limited response in PEM for DERs that can only consume power from the grid. In this work, a linearized virtual battery model of PEM is provided that is capable of predicting the down-ramp limited output of PEM and is used in a model predictive control (MPC) framework to improve the performance of PEM in tracking an automatic generation control (AGC) signal. By performing statistical analysis on the AGC regulation signal, PJM Reg-D, an ARMA model is derived as a predictor for the MPC-based precompensator. Finally, as an alternative to MPC, it is shown that by varying the packet length as a function of time, for example through packet randomization, frequency regulation can be improved under PEM.
We present differentiable predictive control (DPC) as a deep learning-based alternative to the explicit model predictive control (MPC) for unknown nonlinear systems. In the DPC framework, a neural state-space model is learned from time-series measurements of the system dynamics. The neural control policy is then optimized via stochastic gradient descent approach by differentiating the MPC loss function through the closed-loop system dynamics model. The proposed DPC method learns model-based control policies with state and input constraints, while supporting time-varying references and constraints. In embedded implementation using a Raspberry-Pi platform, we experimentally demonstrate that it is possible to train constrained control policies purely based on the measurements of the unknown nonlinear system. We compare the control performance of the DPC method against explicit MPC and report efficiency gains in online computational demands, memory requirements, policy complexity, and construction time. In particular, we show that our method scales linearly compared to exponential scalability of the explicit MPC solved via multiparametric programming.
We present a differentiable predictive control (DPC) methodology for learning constrained control laws for unknown nonlinear systems. DPC poses an approximate solution to multiparametric programming problems emerging from explicit nonlinear model predictive control (MPC). Contrary to approximate MPC, DPC does not require supervision by an expert controller. Instead, a system dynamics model is learned from the observed system's dynamics, and the neural control law is optimized offline by leveraging the differentiable closed-loop system model. The combination of a differentiable closed-loop system and penalty methods for constraint handling of system outputs and inputs allows us to optimize the control law's parameters directly by backpropagating economic MPC loss through the learned system model. The control performance of the proposed DPC method is demonstrated in simulation using learned model of multi-zone building thermal dynamics.
Ensuring nominal asymptotic stability of the Non-linear Model Predictive Control (NMPC) controller is not trivial. Stabilizing ingredients such as terminal penalty term and terminal region are crucial in establishing the asymptotic stability. Approaches available in the literature provide limited degrees of freedom for the characterization of the terminal region for the discrete time Quasi Infinite Horizon NMPC (QIH-NMPC) formulation. Current work presents alternate approaches namely arbitrary controller based approach and LQR based approach, which provide large degrees of freedom for enlarging the terminal region. Both the approaches are scalable to system of any state and input dimension. Approach from the literature provides a scalar whereas proposed approaches provide a linear controller and two additive matrices as tuning parameters for shaping of the terminal region. Proposed approaches involve solving modified Lyapunov equations to compute terminal penalty term, followed by explicit characterization of the terminal region. Efficacy of the proposed approaches is demonstrated using benchmark two state system. Terminal region obtained using the arbitrary controller based approach and LQR based approach are approximately 10.4723 and 9.5055 times larger by area measure when compared to the largest terminal region obtained using the approach from the literature.
Deep reinforcement learning has shown remarkable success in the past few years. Highly complex sequential decision making problems from game playing and robotics have been solved with deep model-free methods. Unfortunately, the sample complexity of model-free methods is often high. To reduce the number of environment samples, model-based reinforcement learning creates an explicit model of the environment dynamics. Achieving high model accuracy is a challenge in high-dimensional problems. In recent years, a diverse landscape of model-based methods has been introduced to improve model accuracy, using methods such as uncertainty modeling, model-predictive control, latent models, and end-to-end learning and planning. Some of these methods succeed in achieving high accuracy at low sample complexity, most do so either in a robotics or in a games context. In this paper, we survey these methods; we explain in detail how they work and what their strengths and weaknesses are. We conclude with a research agenda for future work to make the methods more robust and more widely applicable to other applications.
This paper discusses linearized models of hydropower plants (HPPs). First, it reviews state-of-the-art models and discusses their non-linearities, then it proposes suitable linearization strategies for the plant head, discharge, and turbine torque. It is shown that neglecting the dependency of the hydroacoustic resistance on the discharge leads to a linear formulation of the hydraulic circuits model. For the turbine, a numerical linearization based on a first-order Taylor expansion is proposed. Model performance is evaluated for a medium- and a low-head HPP with a Francis and Kaplan turbine, respectively. Perspective applications of these linear models are in the context of efficient model predictive control of HPPs based on convex optimization.
This paper aims to improve the reliability of optimal control using models constructed by machine learning methods. Optimal control problems based on such models are generally non-convex and difficult to solve online. In this paper, we propose a model that combines the Hammerstein-Wiener model with input convex neural networks, which have recently been proposed in the field of machine learning. An important feature of the proposed model is that resulting optimal control problems are effectively solvable exploiting their convexity and partial linearity while retaining flexible modeling ability. The practical usefulness of the method is examined through its application to the modeling and control of an engine airpath system.
We describe a system for visually guided autonomous navigation of under-canopy farm robots. Low-cost under-canopy robots can drive between crop rows under the plant canopy and accomplish tasks that are infeasible for over-the-canopy drones or larger agricultural equipment. However, autonomously navigating them under the canopy presents a number of challenges: unreliable GPS and LiDAR, high cost of sensing, challenging farm terrain, clutter due to leaves and weeds, and large variability in appearance over the season and across crop types. We address these challenges by building a modular system that leverages machine learning for robust and generalizable perception from monocular RGB images from low-cost cameras, and model predictive control for accurate control in challenging terrain. Our system, CropFollow, is able to autonomously drive 485 meters per intervention on average, outperforming a state-of-the-art LiDAR based system (286 meters per intervention) in extensive field testing spanning over 25 km.
The hybrid electric system has good potential for unmanned tracked vehicles due to its excellent power and economy. Due to unmanned tracked vehicles have no traditional driving devices, and the driving cycle is uncertain, it brings new challenges to conventional energy management strategies. This paper proposes a novel energy management strategy for unmanned tracked vehicles based on local speed planning. The contributions are threefold. Firstly, a local speed planning algorithm is adopted for the input of driving cycle prediction to avoid the dependence of traditional vehicles on driver's operation. Secondly, a prediction model based on Convolutional Neural Networks and Long Short-Term Memory (CNN-LSTM) is proposed, which is used to process both the planned and the historical velocity series to improve the prediction accuracy. Finally, based on the prediction results, the model predictive control algorithm is used to realize the real-time optimization of energy management. The validity of the method is verified by simulation using collected data from actual field experiments of our unmanned tracked vehicle. Compared with multi-step neural networks, the prediction model based on CNN-LSTM improves the prediction accuracy by 20%. Compared with the traditional regular energy management strategy, the energy management strategy based on model predictive control reduces fuel consumption by 7%.
Electron beam stabilization in a synchrotron is a disturbance rejection problem, with hundreds of inputs and outputs, that is sampled at frequencies higher than $10$ kHz. In this feasibility study, we focus on the practical issues of an efficient implementation of model predictive control (MPC) for the heavily ill-conditioned plant of the electron beam stabilization problem. To obtain a tractable control problem that can be solved using only a few iterations of the fast gradient method, we investigate different methods for preconditioning the resulting optimization problem and relate our findings to standard regularization techniques from cross-directional control. We summarize the single- and multi-core implementations of our control algorithm on a digital signal processor (DSP), and show that MPC can be executed at the rate required for synchrotron control. MPC overcomes various problems of standard electron beam stabilization techniques, and the successful implementation can increase the stability of photon beams in synchrotron light sources.
Trajectory planning in urban automated driving is challenging because of the high uncertainty resulting from the unknown future motion of other traffic participants. Robust approaches guarantee safety, but tend to result in overly conservative motion planning. Hence, we propose to use Stochastic Model Predictive Control for vehicle control in urban driving, allowing to efficiently plan the vehicle trajectory, while maintaining the risk probability sufficiently low. For motion optimization, we propose to use a two-stage hierarchical structure that plans the trajectory and the maneuver separately. A high-level layer takes advantage of a long prediction horizon and of an abstract model to plan the optimal maneuver, and a lower level is in charge of executing the selected maneuver by properly planning the vehicle's trajectory. Numerical simulations are included, showing the potential of our proposal.
Based on equivalent-dynamic-linearization model (EDLM), we propose a kind of model predictive control (MPC) for single-input and single-output (SISO) nonlinear or linear systems. After compensating the EDLM with disturbance for multiple-input and multiple-output nonlinear or linear systems, the MPC compensated with disturbance is proposed to address the disturbance rejection problem. The system performance analysis results are much clear compared with the system stability analyses on MPC in current works. And this may help the engineers understand how to design, analyze and apply the controller in practical.
Building energy management is one of the core problems in modern power grids to reduce energy consumption while ensuring occupants' comfort. However, the building energy management system (BEMS) is now facing more challenges and uncertainties with the increasing penetration of renewables and complicated interactions between humans and buildings. Classical model predictive control (MPC) has shown its capacity to reduce building energy consumption, but it suffers from labor-intensive modelling and complex on-line control optimization. Recently, with the growing accessibility to the building control and automation data, data-driven solutions have attracted more research interest. This paper presents a compact review of the recent advances in data-driven MPC and reinforcement learning based control methods for BEMS. The main challenges in these approaches and insights on the selection of a control method are discussed.
In this paper, we present a data-driven Model Predictive Controller that leverages a Gaussian Process to generate optimal motion policies for connected autonomous vehicles in regions with uncertainty in the wireless channel. The communication channel between the vehicles of a platoon can be easily influenced by numerous factors, e.g. the surrounding environment, and the relative states of the connected vehicles, etc. In addition, the trajectories of the vehicles depend significantly on the motion policies of the preceding vehicle shared via the wireless channel and any delay can impact the safety and optimality of its performance. In the presented algorithm, Gaussian Process learns the wireless channel model and is involved in the Model Predictive Controller to generate a control sequence that not only minimizes the conventional motion costs, but also minimizes the estimated delay of the wireless channel in the future. This results in a farsighted controller that maximizes the amount of transferred information beyond the controller's time horizon, which in turn guarantees the safety and optimality of the generated trajectories in the future. To decrease computational cost, the algorithm finds the reachable set from the current state and focuses on that region to minimize the size of the kernel matrix and related calculations. In addition, we present an efficient recursive approach to decrease the time complexity of developing the data-driven model and involving it in Model Predictive Control. We demonstrate the capability of the presented algorithm in a simulated scenario.