We consider estimation and control in linear time-varying dynamical systems from the perspective of regret minimization. Unlike most prior work in this area, we focus on the problem of designing causal estimators and controllers which compete against a clairvoyant noncausal policy, instead of the best policy selected in hindsight from some fixed parametric class. We show that the regret-optimal estimator and regret-optimal controller can be derived in state-space form using operator-theoretic techniques from robust control and present tight,data-dependent bounds on the regret incurred by our algorithms in terms of the energy of the disturbances. Our results can be viewed as extending traditional robust estimation and control, which focuses on minimizing worst-case cost, to minimizing worst-case regret. We propose regret-optimal analogs of Model-Predictive Control (MPC) and the Extended KalmanFilter (EKF) for systems with nonlinear dynamics and present numerical experiments which show that our regret-optimal algorithms can significantly outperform standard approaches to estimation and control.
Economic model predictive control (EMPC) has attracted significant attention in recent years and is recognized as a promising advanced process control method for the next generation smart manufacturing. It can lead to improving economic performance but at the same time increases the computational complexity significantly. Model approximation has been a standard approach for reducing computational complexity in process control. In this work, we perform a study on three types of representative model approximation methods applied to EMPC, including model reduction based on available first-principle models (e.g., proper orthogonal decomposition), system identification based on input-output data (e.g., subspace identification) that results in an explicitly expressed mathematical model, and neural networks based on input-output data. A representative algorithm from each model approximation method is considered. Two processes that are very different in dynamic nature and complexity were selected as benchmark processes for computational complexity and economic performance comparison, namely an alkylation process and a wastewater treatment plant (WWTP). The strengths and drawbacks of each method are summarized according to the simulation results, with future research direction regarding control oriented model approximation proposed at the end.
Cloud computing creates new possibilities for control applications by offering powerful computation and storage capabilities. In this paper, we propose a novel cloud-assisted model predictive control (MPC) framework in which we systematically fuse a cloud MPC that uses a high-fidelity nonlinear model but is subject to communication delays with a local MPC that exploits simplified dynamics (due to limited computation) but has timely feedback. Unlike traditional cloud-based control that treats the cloud as powerful, remote, and sole controller in a networked-system control setting, the proposed framework aims at seamlessly integrating the two controllers for enhanced performance. In particular, we formalize the fusion problem for finite-duration tasks by explicitly considering model mismatches and errors due to request-response communication delays. We analyze stability-like properties of the proposed cloud-assisted MPC framework and establish approaches to robustly handling constraints within this framework in spite of plant-model mismatch and disturbances. A fusion scheme is then developed to enhance control performance while satisfying stability-like conditions, the efficacy of which is demonstrated with multiple simulation examples, including an automotive control example to show its industrial application potentials.
Simultaneous vibration control and energy harvesting of vehicle suspensions have attracted significant research attention over the past decades. However, existing energy harvesting shock absorbers (EHSAs) are mainly designed based on the principle of linear resonance, thereby compromising suspension performance for high-efficiency energy harvesting and being only responsive to narrow bandwidth vibrations. In this paper, we propose a new EHSA design -- inerter pendulum vibration absorber (IPVA) -- that integrates an electromagnetic rotary EHSA with a nonlinear pendulum vibration absorber. We show that this design simultaneously improves ride comfort and energy harvesting efficiency by exploiting the nonlinear effects of pendulum inertia. To further improve the performance, we develop a novel stochastic linearization model predictive control (SL-MPC) approach in which we employ stochastic linearization to approximate the nonlinear dynamics of EHSA that has superior accuracy compared to standard linearization. In particular, we develop a new stochastic linearization method with guaranteed stabilizability, which is a prerequisite for control designs. This leads to an MPC problem that is much more computationally efficient than the nonlinear MPC counterpart with no major performance degradation. Extensive simulations are performed to show the superiority of the proposed new nonlinear EHSA and to demonstrate the efficacy of the proposed SL-MPC.
In this paper, we provide a stability and performance analysis of model predictive control (MPC) schemes based on finite-tail costs. We study the MPC formulation originally proposed by Magni et al. (2001) wherein the standard terminal penalty is replaced by a finite-horizon cost of some stabilizing control law. In order to analyse the closed loop, we leverage the more recent technical machinery developed for MPC without terminal ingredients. For a specified set of initial conditions, we obtain sufficient conditions for stability and a performance bound in dependence of the prediction horizon and the extended horizon used for the terminal penalty. The main practical benefit of the considered finite-tail cost MPC formulation is the simpler offline design in combination with typically significantly less restrictive bounds on the prediction horizon to ensure stability. We demonstrate the benefits of the considered MPC formulation using the classical example of a four tank system.
We develop a learning-based control algorithm for unknown dynamical systems under very severe data limitations. Specifically, the algorithm has access to streaming and noisy data only from a single and ongoing trial. It accomplishes such performance by effectively leveraging various forms of side information on the dynamics to reduce the sample complexity. Such side information typically comes from elementary laws of physics and qualitative properties of the system. More precisely, the algorithm approximately solves an optimal control problem encoding the system's desired behavior. To this end, it constructs and iteratively refines a data-driven differential inclusion that contains the unknown vector field of the dynamics. The differential inclusion, used in an interval Taylor-based method, enables to over-approximate the set of states the system may reach. Theoretically, we establish a bound on the suboptimality of the approximate solution with respect to the optimal control with known dynamics. We show that the longer the trial or the more side information is available, the tighter the bound. Empirically, experiments in a high-fidelity F-16 aircraft simulator and MuJoCo's environments illustrate that, despite the scarcity of data, the algorithm can provide performance comparable to reinforcement learning algorithms trained over millions of environment interactions. Besides, we show that the algorithm outperforms existing techniques combining system identification and model predictive control.
Economic Model Predictive Control has recently gained popularity due to its ability to directly optimize a given performance criterion, while enforcing constraint satisfaction for nonlinear systems. Recent research has developed both numerical algorithms and stability analysis for the undiscounted case. The introduction of a discount factor in the cost, however, can be desirable in some cases of interest, e.g., economics, stochastically terminating processes, Markov decision processes, etc. Unfortunately, the stability theory in this case is still not fully developed. In this paper we propose a new dissipativity condition to prove asymptotic stability in the infinite horizon case and we connect our results with existing ones in the literature on discounted economic optimal control. Numerical examples are provided to illustrate the theoretical results.
Cooperative driving, enabled by communication between automated vehicle systems, is expected to significantly contribute to transportation safety and efficiency. Cooperative Adaptive Cruise Control (CACC) and platooning are two of the main cooperative driving applications that are currently under study. These applications offer significant improvements over current advanced driver assistant systems such as adaptive cruise control (ACC). The primary motivation of CACC and Platooning is to reduce traffic congestion and improve traffic flow, traffic throughput, and highway capacity. These applications need an efficient controller to consider the computational cost and ensure driving comfort and high responsiveness. The advantage of Model Predictive Control is that we can realize high control performance since all constrain for these applications can be explicitly dealt with through solving an optimization problem. These applications highly depend on information update and Communication reliability for their safety and stability purposes. In this paper, we propose a Model Predictive Control (MPC) based approach for CACC and platooning, and examine the impact of communication loss on the performance and robustness of the control scheme. The results show an improvement in response time and string stability, demonstrating the potential of cooperation to attenuate disturbances and improve traffic flow.
In this work, we propose a Model Predictive Control (MPC)-based Reinforcement Learning (RL) method for Autonomous Surface Vehicles (ASVs). The objective is to find an optimal policy that minimizes the closed-loop performance of a simplified freight mission, including collision-free path following, autonomous docking, and a skillful transition between them. We use a parametrized MPC-scheme to approximate the optimal policy, which considers path-following/docking costs and states (position, velocity)/inputs (thruster force, angle) constraints. The Least Squares Temporal Difference (LSTD)-based Deterministic Policy Gradient (DPG) method is then applied to update the policy parameters. Our simulation results demonstrate that the proposed MPC-LSTD-based DPG method could improve the closed-loop performance during learning for the freight mission problem of ASV.
Model Predictive Control (MPC) has shown to be a successful method for many applications that require control. Especially in the presence of prediction uncertainty, various types of MPC offer robust or efficient control system behavior. For modeling, uncertainty is most often approximated in such a way that established MPC approaches are applicable for specific uncertainty types. However, for a number of applications, especially automated vehicles, uncertainty in predicting the future behavior of other agents is more suitably modeled by a twofold description: a high-level task uncertainty and a low-level execution uncertainty of individual tasks. In this work, we present an MPC framework that is capable of dealing with this twofold uncertainty. A scenario MPC approach considers the possibility of other agents performing one of multiple tasks, with an arbitrary probability distribution, while an analytic stochastic MPC method handles execution uncertainty within a specific task, based on a Gaussian distribution. Combining both approaches allows to efficiently handle the twofold uncertainty structure of many applications. Application of the proposed MPC method is demonstrated in an automated vehicle simulation study.
While Robust Model Predictive Control considers the worst-case system uncertainty, Stochastic Model Predictive Control, using chance constraints, provides less conservative solutions by allowing a certain constraint violation probability depending on a predefined risk parameter. However, for safety-critical systems it is not only important to bound the constraint violation probability but to reduce this probability as much as possible. Therefore, an approach is necessary that minimizes the constraint violation probability while ensuring that the Model Predictive Control optimization problem remains feasible. We propose a novel Model Predictive Control scheme that yields a solution with minimal constraint violation probability for a norm constraint in an environment with uncertainty. After minimal constraint violation is guaranteed the solution is then also optimized with respect to other control objectives. Further, it is possible to account for changes over time of the support of the uncertainty. We first present a general method and then provide an approach for uncertainties with symmetric, unimodal probability density function. Recursive feasibility and convergence of the method are proved. A simulation example demonstrates the effectiveness of the proposed method.
This paper proposes a fuel-economical distributed model predictive control design (Eco-DMPC) for a homogenous heavy-duty truck platoon. The proposed control strategy integrates a fuel-optimal control strategy for the leader truck with a distributed formation control for the following trucks in the heavy-duty truck platoon. The fuel-optimal control strategy is implemented by a nonlinear model predictive control (NMPC) design with an instantaneous fuel consumption model. The proposed fuel-optimal control strategy utilizes the preview information of the preceding traffic to achieve the fuel-economical speed planning by avoiding energy-inefficient maneuvers, particularly under transient traffic conditions. The distributed formation control is designed with a serial distributed model predictive control (DMPC) strategy with guaranteed local and string stability. In the DMPC strategy, each following truck acquires the future predicted state information of its predecessor through vehicle connectivity and then applies local optimal control to maintain constant spacing. Simulation studies are conducted to investigate the fuel economy performance of the proposed control strategy and to validate the local and string stability of the platoon under a realistic traffic scenario. Compared with a human-operated platoon and a benchmark formation-controlled platoon, the proposed Eco-DMPC significantly improves the fuel economy and road utilization.
This paper presents an approach to deal with safety of dynamical systems in presence of multiple non-convex unsafe sets. While optimal control and model predictive control strategies can be employed in these scenarios, they suffer from high computational complexity in case of general nonlinear systems. Leveraging control barrier functions, on the other hand, results in computationally efficient control algorithms. Nevertheless, when safety guarantees have to be enforced alongside stability objectives, undesired asymptotically stable equilibrium points have been shown to arise. We propose a computationally efficient optimization-based approach which allows us to ensure safety of dynamical systems without introducing undesired equilibria even in presence of multiple non-convex unsafe sets. The developed control algorithm is showcased in simulation and in a real robot navigation application.
This paper studies the formation stabilization problem of asynchronous nonlinear multi-agent systems (MAS) subject to parametric uncertainties, external disturbances and bounded time-varying communication delays. A self-triggered min-max distributed model predictive control (DMPC) approach is proposed to handle these practical issues. At triggering instants, each agent solves a local min-max optimization problem based on local system states and predicted system states of neighbors, determines its next triggering instant and broadcasts its predicted state trajectory to its neighbors. As a result, the communication load is greatly alleviated while retaining robustness and comparable control performance compared to periodic algorithms. In order to handle time-varying delays, a novel consistency constraint is incorporated into each local optimization problem to restrict the deviation between the newest predicted states and previously broadcast predicted states. Consequently, each agent can utilize previously predicted states of its neighbors to achieve cooperation in the presence of time-varying delays and asynchronous communication induced by the distributed triggered scheduler. The recursive feasibility of the proposed algorithm and the closed-loop stability of MAS at triggering time instants are proven. Finally, numerical simulations are conducted to verify the efficiency of the proposed control method.
In this paper, we present the use of Model Predictive Control (MPC) based on Reinforcement Learning (RL) to find the optimal policy for a multi-agent battery storage system. A time-varying prediction of the power price and production-demand uncertainty are considered. We focus on optimizing an economic objective cost while avoiding very low or very high state of charge, which can damage the battery. We consider the bounded power provided by the main grid and the constraints on the power input and state of each agent. A parametrized MPC-scheme is used as a function approximator for the deterministic policy gradient method and RL optimizes the closed-loop performance by updating the parameters. Simulation results demonstrate that the proposed method is able to tackle the constraints and deliver the optimal policy.
This paper addresses the issue of power flow control for partially faulty microgrids. In microgrid control systems, faults may occur in both electrical and communication layers. This may have severe effects on the operation of microgrids. In addition, disturbances always coexist with faults in microgrids, which may further deteriorate system performance. To address the faults and disturbances simultaneously, a model predictive control (MPC) method based on set-membership estimation (SME) that transmits information via a communication network is proposed. When electrical devices are nonfunctional or communication failures occur, the corresponding system states will become unavailable. To this end, the SME method is employed to estimate the states with the existence of unknown-but-bounded process and measurement disturbances. The networked MPC method is designed to schedule the power dispatch by using the forecasts of photovoltaic (PV) generation and load demand. With these two methods, the fault-tolerant control can be achieved. Further, a deviation compensation method is proposed to compensate for the forecast errors. The effectiveness of the proposed control strategy is demonstrated through wireless communication tests using Raspberry Pis.
This paper presents a model predictive control (MPC)-based online real-time adaptive control scheme for emergency voltage control in power systems. Despite tremendous success in various applications, real-time implementation of MPC for control in power systems has not been successful due to its online computational burden for large-sized systems that takes more time than available between the two control decisions. This long-standing problem is addressed here by developing a novel MPC-based adaptive control framework which (i) adapts the nominal offline computed control, by successive control corrections, at each control decision point using the latest measurements, (ii) utilizes data-driven approach for prediction of voltage trajectory and its sensitivity with respect to control using trained deep neural networks (DNNs). In addition, a realistic coordination scheme among control inputs of static var compensators (SVC), load-shedding (LS), and load tap-changers (LTC) is presented with a goal of maintaining bus voltages within a predefined permissible range, where the delayed effect of LTC action is also incorporated in a novel way. The performance of the proposed scheme is validated for IEEE 9-bus as well as 39-bus systems, with $\pm 20\%$ variations in nominal loading conditions. We also show that the proposed new scheme speeds up the online computation by a factor of 20 bringing it down to under one-tenth the control interval, making the MPC-based power system control practically feasible.
Load serving entities with storage units reach sizes and performances that can significantly impact clearing prices in electricity markets. Nevertheless, price endogeneity is rarely considered in storage bidding strategies and modeling the electricity market is a challenging task. Meanwhile, model-free reinforcement learning such as the Actor-Critic are becoming increasingly popular for designing energy system controllers. Yet implementation frequently requires lengthy, data-intense, and unsafe trial-and-error training. To fill these gaps, we implement an online Supervised Actor-Critic (SAC) algorithm, supervised with a model-based controller -- Model Predictive Control (MPC). The energy storage agent is trained with this algorithm to optimally bid while learning and adjusting to its impact on the market clearing prices. We compare the supervised Actor-Critic algorithm with the MPC algorithm as a supervisor, finding that the former reaps higher profits via learning. Our contribution, thus, is an online and safe SAC algorithm that outperforms the current model-based state-of-the-art.
Continuous integration of renewable energy sources into power networks is causing a paradigm shift in energy generation and distribution with regards to trading and control; the intermittent nature of renewable sources affects pricing of energy sold or purchased; the networks are subject to operational constraints, voltage limits at each node, rated capacities for the power electronic devices, current bounds for distribution lines. These economic and technical constraints coupled with intermittent renewable injection may pose a threat to system stability and performance. We propose a novel holistic approach to energy trading composed of a distributed predictive control framework to handle physical interactions, i,e., voltage constraints and power dispatch, together with a negotiation framework to determine pricing policies for energy transactions. We study the effect of forecasting generation and consumption on the overall network's performance and market behaviours. We provide a rigorous convergence analysis for both the negotiation framework and the distributed control. Lastly, we assess the impact of forecasting in the proposed system with the aid of testing scenarios.
Quadcopters are increasingly used for applications ranging from hobby to industrial products and services. This paper serves as a tutorial on the design, simulation, implementation, and experimental outdoor testing of digital quadcopter flight controllers, including Explicit Model Predictive Control, Linear Quadratic Regulator, and Proportional Integral Derivative. A quadcopter was flown in an outdoor testing facility and made to track an inclined, circular path at different tangential velocities under ambient wind conditions. Controller performance was evaluated via multiple metrics, such as position tracking error, velocity tracking error, and onboard computation time. Challenges related to the use of computationally limited embedded hardware and flight in an outdoor environment are addressed with proposed solutions.