This paper proposes and experimentally validates a joint control and scheduling framework for a grid-forming converter-interfaced BESS providing multiple services to the electrical grid. The framework is designed to dispatch the operation of a distribution feeder hosting heterogeneous prosumers according to a dispatch plan and provide frequency containment reserve and voltage control as additional services. The framework consists of three phases. In the day-ahead scheduling phase, a robust optimization problem is solved to compute the optimal dispatch plan and frequency droop coefficient, accounting for the uncertainty of the aggregated prosumption. In the intra-day phase, a model predictive control algorithm is used to compute the power set-point for the BESS to achieve the tracking of the dispatch plan. Finally, in a real-time stage, the power setpoint originated by the dispatch tracking is converted into a feasible frequency set-point for the grid forming converter by means of a convex optimisation problem accounting for the capability curve of the power converter. The proposed framework is experimentally validated by using a grid-scale 720 kVA/560 kWh BESS connected to a 20 kV distribution feeder of the EPFL hosting stochastic prosumption and PV generation.
Recent advances in latent space dynamics model from pixels show promising progress in vision-based model predictive control (MPC). However, executing MPC in real time can be challenging due to its intensive computational cost in each timestep. We propose to introduce additional learning objectives to enforce that the learned latent space is proportional derivative controllable. In execution time, the simple PD-controller can be applied directly to the latent space encoded from pixels, to produce simple and effective control to systems with visual observations. We show that our method outperforms baseline methods to produce robust goal reaching and trajectory tracking in various environments.
Model predictive control (MPC) has been used widely in power electronics due to its simple concept, fast dynamic response, and good reference tracking. However, it suffers from parametric uncertainties, since it directly relies on the mathematical model of the system to predict the optimal switching states to be used at the next sampling time. As a result, uncertain parameters lead to an ill-designed MPC. Thus, this paper offers a model-free control strategy on the basis of artificial neural networks (ANNs), for mitigating the effects of parameter mismatching while having a little negative impact on the inverter's performance. This method includes two related stages. First, MPC is used as an expert to control the studied converter in order to provide a dataset, while, in the second stage, the obtained dataset is utilized to train the proposed ANN. The case study herein is based on a four-level three-cell flying capacitor inverter. In this study, MATLAB/Simulink is used to simulate the performance of the proposed method, taking into account various operating conditions. Afterward, the simulation results are reported in comparison with the conventional MPC scheme, demonstrating the superior performance of the proposed control strategy in terms of robustness against parameters mismatch and low total harmonic distortion (THD), especially when changes occur in the system parameters, compared to the conventional MPC. Furthermore, the experimental validation of the proposed method is provided based on the Hardware-in-the-Loop (HIL) simulation using the C2000TM-microcontroller-LaunchPadXL TMS320F28379D kit, demonstrating the applicability of the ANN-based control strategy to be implemented on a DSP controller.
Hyperparameter optimization (HPO) is generally treated as a bi-level optimization problem that involves fitting a (probabilistic) surrogate model to a set of observed hyperparameter responses, e.g. validation loss, and consequently maximizing an acquisition function using a surrogate model to identify good hyperparameter candidates for evaluation. The choice of a surrogate and/or acquisition function can be further improved via knowledge transfer across related tasks. In this paper, we propose a novel transfer learning approach, defined within the context of model-based reinforcement learning, where we represent the surrogate as an ensemble of probabilistic models that allows trajectory sampling. We further propose a new variant of model predictive control which employs a simple look-ahead strategy as a policy that optimizes a sequence of actions, representing hyperparameter candidates to expedite HPO. Our experiments on three meta-datasets comparing to state-of-the-art HPO algorithms including a model-free reinforcement learning approach show that the proposed method can outperform all baselines by exploiting a simple planning-based policy.
In this paper, we present the development and deployment of an embedded optimal control strategy for autonomous driving applications on a Ford Focus road vehicle. Non-linear model predictive control (NMPC) is designed and deployed on a system with hard real-time constraints. We show the properties of sequential quadratic programming (SQP) optimization solvers that are suitable for driving tasks. Importantly, the designed algorithms are validated based on a standard automotive XiL development cycle: model-in-the-loop (MiL) with high fidelity vehicle dynamics, hardware-in-the-loop (HiL) with vehicle actuation and embedded platform, and full vehicle-hardware-in-the-loop (VeHiL). The autonomous driving environment contains both virtual simulation and physical proving ground tracks. NMPC algorithms and optimal control problem formulation are fine-tuned using a deployable C code via code generation compatible with the target embedded toolchains. Finally, the developed systems are applied to autonomous collision avoidance, trajectory tracking, and lane change at high speed on city/highway and low speed at a parking environment.
There is a widespread intuition that model-based control methods should be able to surpass the data efficiency of model-free approaches. In this paper we attempt to evaluate this intuition on various challenging locomotion tasks. We take a hybrid approach, combining model predictive control (MPC) with a learned model and model-free policy learning; the learned policy serves as a proposal for MPC. We find that well-tuned model-free agents are strong baselines even for high DoF control problems but MPC with learned proposals and models (trained on the fly or transferred from related tasks) can significantly improve performance and data efficiency in hard multi-task/multi-goal settings. Finally, we show that it is possible to distil a model-based planner into a policy that amortizes the planning computation without any loss of performance. Videos of agents performing different tasks can be seen at https://sites.google.com/view/mbrl-amortization/home.
In this work we address the problem of performing a repetitive task when we have uncertain observations and dynamics. We formulate this problem as an iterative infinite horizon optimal control problem with output feedback. Previously, this problem was solved for linear time-invariant (LTI) system for the case when noisy full-state measurements are available using a robust iterative learning control framework, which we refer to as robust learning-based model predictive control (RL-MPC). However, this work does not apply to the case when only noisy observations of part of the state are available. This limits the applicability of current approaches in practice: First, in practical applications we typically do not have access to the full state. Second, uncertainties in the observations, when not accounted for, can lead to instability and constraint violations. To overcome these limitations, we propose a combination of RL-MPC with robust output feedback model predictive control, named robust learning-based output feedback model predictive control (RLO-MPC). We show recursive feasibility and stability, and prove theoretical guarantees on the performance over iterations. We validate the proposed approach with a numerical example in simulation and a quadrotor stabilization task in experiments.
In this paper, the disjunctive and conjunctive lattice piecewise affine (PWA) approximations of explicit linear model predictive control (MPC) are proposed. The training data are generated uniformly in the domain of interest, consisting of the state samples and corresponding affine control laws, based on which the lattice PWA approximations are constructed. Re-sampling of data is also proposed to guarantee that the lattice PWA approximations are identical to explicit MPC control law in the unique order (UO) regions containing the sample points as interior points. Additionally, under mild assumptions, the equivalence of the two lattice PWA approximations guarantees that the approximations are error-free in the domain of interest. The algorithms for deriving statistically error-free approximation to the explicit linear MPC are proposed and the complexity of the entire procedure is analyzed, which is polynomial with respect to the number of samples. The performance of the proposed approximation strategy is tested through two simulation examples, and the result shows that with a moderate number of sample points, we can construct lattice PWA approximations that are equivalent to optimal control law of the explicit linear MPC.
While distributed algorithms provide advantages for the control of complex large-scale systems by requiring a lower local computational load and less local memory, it is a challenging task to design high-performance distributed control policies. Learning-based control algorithms offer promising opportunities to address this challenge, but generally cannot guarantee safety in terms of state and input constraint satisfaction. A recently proposed safety framework for centralized linear systems ensures safety by matching the learning-based input online with the initial input of a model predictive control law capable of driving the system to a terminal set known to be safe. We extend this idea to derive a distributed model predictive safety certification (DMPSC) scheme, which is able to ensure state and input constraint satisfaction when applying any learning-based control algorithm to an uncertain distributed linear system with dynamic couplings. The scheme is based on a distributed tube-based model predictive control (MPC) concept, where subsystems negotiate local tube sizes among neighbors in order to mitigate restrictiveness of the safety approach. In addition, we present a technique for generating a structured ellipsoidal robust positive invariant tube. In numerical simulations, we show that the safety framework ensures constraint satisfaction for an initially unsafe control policy and allows to improve overall control performance compared to robust distributed MPC.
This paper deals with the integration of input move-blocking into the framework of suboptimal model predictive control. The blocked input parameterization is explicitly considered as a source of suboptimality. A straightforward integration approach is to hold back a manually generated stabilizing fallback solution in some buffer for the case that the optimizer does not find a better input move-blocked solution. An extended approach superimposes the manually generated stabilizing warm-start by the move-blocked control sequence and enables a stepwise improvement of the control performance. In addition, this contribution provides a detailed review of the literature on input move-blocked model predictive control and combines important results with the findings of suboptimal model predictive control. A numerical example supports the theoretical results and shows the effectiveness of the proposed approach.
This tutorial shows an overview of Model Predictive Control with a linear discrete-time system and constrained states and inputs. The focus is on the implementation of the method under consideration of stability and recursive feasibility. The MATLAB code for the examples and plots is available online.
Nonlinear Robust Model Predictive Control (RMPC) provides a very promising solution to the problem of automatic emergency maneuvering, which is capable of handling multiple possibly conflicting objectives of robustness and performance. Even though RMPC gives a suboptimal solution, the key challenge in real-time implementation is that it is computationally very demanding. In this paper a real-time capable robust tube MPC based framework for steering control during emergency obstacle avoidance maneuver is presented. The novelty of this framework lies in the robust integration of path planning and path following tasks of autonomous vehicles. A simulation study showcases the robust performance improvements due to the proposed strategy over a non-robust MPC in different extreme maneuvering scenarios.
We discuss nonlinear model predictive control (NMPC) for multi-body dynamics via physics-informed machine learning methods. Physics-informed neural networks (PINNs) are a promising tool to approximate (partial) differential equations. PINNs are not suited for control tasks in their original form since they are not designed to handle variable control actions or variable initial values. We thus present the idea of enhancing PINNs by adding control actions and initial conditions as additional network inputs. The high-dimensional input space is subsequently reduced via a sampling strategy and a zero-hold assumption. This strategy enables the controller design based on a PINN as an approximation of the underlying system dynamics. The additional benefit is that the sensitivities are easily computed via automatic differentiation, thus leading to efficient gradient-based algorithms. Finally, we present our results using our PINN-based MPC to solve a tracking problem for a complex mechanical system, a multi-link manipulator.
This paper presents a robust economic model predictive control (EMPC) formulation with zone tracking for discrete-time uncertain nonlinear systems. The proposed design ensures that the zone tracking objective is achieved in finite steps and at the same time optimizes the economic performance. In the proposed design, instead of tracking the original target zone, a robust control invariant set within the target zone is determined and is used as the actual zone tracked in the proposed EMPC. This approach ensures that the zone tracking objective is achieved within finite steps and once the zone tracking objective is achieved (the system state enters the robust control invariant set), the system state does not come out of the target zone anymore. To optimize the economic performance within the zone in the presence of disturbances, we introduce the notion of risk factor in the controller design. An algorithm to determine the economic zone to be tracked is provided. The risk factor determines the conservativeness of the controller and provides a way to tune the EMPC for better economic performance. A nonlinear chemical example is presented to demonstrate the performance of the proposed formulation.
Safe UAV navigation is challenging due to the complex environment structures, dynamic obstacles, and uncertainties from measurement noises and unpredictable moving obstacle behaviors. Although plenty of recent works achieve safe navigation in complex static environments with sophisticated mapping algorithms, such as occupancy map and ESDF map, these methods cannot reliably handle dynamic environments due to the mapping limitation from moving obstacles. To address the limitation, this paper proposes a trajectory planning framework to achieve safe navigation considering complex static environments with dynamic obstacles. To reliably handle dynamic obstacles, we divide the environment representation into static mapping and dynamic object representation, which can be obtained from computer vision methods. Our framework first generates a static trajectory based on the proposed iterative corridor shrinking algorithm. Then, reactive chance-constrained model predictive control with temporal goal tracking is applied to avoid dynamic obstacles with uncertainties. The simulation results in various environments demonstrate the ability of our algorithm to navigate safely in complex static environments with dynamic obstacles.
Adaptive model predictive control (MPC) robustly ensures safety while reducing uncertainty during operation. In this paper, a distributed version is proposed to deal with network systems featuring multiple agents and limited communication. To solve the problem in a distributed manner, structure is imposed on the control design ingredients without sacrificing performance. Decentralized and distributed adaptation schemes that allow for a reduction of the uncertainty online compatibly with the network topology are also proposed. The algorithm ensures robust constraint satisfaction, recursive feasibility and finite gain $\ell_2$ stability, and yields lower closed-loop cost compared to robust distributed MPC in simulations.
Motion planning for autonomous robots and vehicles in presence of uncontrolled agents remains a challenging problem as the reactive behaviors of the uncontrolled agents must be considered. Since the uncontrolled agents usually demonstrate multimodal reactive behavior, the motion planner needs to solve a continuous motion planning problem under these behaviors, which contains a discrete element. We propose a branch Model Predictive Control (MPC) framework that plans over feedback policies to leverage the reactive behavior of the uncontrolled agent. In particular, a scenario tree is constructed from a finite set of policies of the uncontrolled agent, and the branch MPC solves for a feedback policy in the form of a trajectory tree, which shares the same topology as the scenario tree. Moreover, coherent risk measures such as the Conditional Value at Risk (CVaR) are used as a tuning knob to adjust the tradeoff between performance and robustness. The proposed branch MPC framework is tested on an overtake and lane change task and a merging task for autonomous vehicles in simulation, and on the motion planning of an autonomous quadruped robot alongside an uncontrolled quadruped in experiments. The result demonstrates interesting human-like behaviors, achieving a balance between safety and performance.
Model Predictive Control (MPC) formulations are typically built on the requirement that a feasible reference trajectory is available. In practical settings, however, references that are infeasible with respect to the system dynamics are used for convenience. In this paper, we prove under which conditions an MPC formulation is Input-to-State Stable~(ISS) in closed-loop when an infeasible reference is used, and that with proper terminal conditions, asymptotic stability towards an optimal reference may be achieved. We illustrate the theoretical results with a four-dimensional robotic joint example.
In this work, we consider the problem of deriving and incorporating accurate dynamic models for model predictive control (MPC) with an application to quadrotor control. MPC relies on precise dynamic models to achieve the desired closed-loop performance. However, the presence of uncertainties in complex systems and the environments they operate in poses a challenge in obtaining sufficiently accurate representations of the system dynamics. In this work, we make use of a deep learning tool, knowledge-based neural ordinary differential equations (KNODE), to augment a model obtained from first principles. The resulting hybrid model encompasses both a nominal first-principle model and a neural network learnt from simulated or real-world experimental data. Using a quadrotor, we benchmark our hybrid model against a state-of-the-art Gaussian Process (GP) model and show that the hybrid model provides more accurate predictions of the quadrotor dynamics and is able to generalize beyond the training data. To improve closed-loop performance, the hybrid model is integrated into a novel MPC framework, known as KNODE-MPC. Results show that the integrated framework achieves 60.2% improvement in simulations and more than 21% in physical experiments, in terms of trajectory tracking performance.
Understanding how to effectively control an epidemic spreading on a network is a problem of paramount importance for the scientific community. The ongoing COVID-19 pandemic has highlighted the need for policies that mitigate the spread, without relying on pharmaceutical interventions, that is, without the medical assurance of the recovery process. These policies typically entail lockdowns and mobility restrictions, having thus nonnegligible socio-economic consequences for the population. In this paper, we focus on the problem of finding the optimum policies that "flatten the epidemic curve" while limiting the negative consequences for the society, and formulate it as a nonlinear control problem over a finite prediction horizon. We utilize the model predictive control theory to design a strategy to effectively control the disease, balancing safety and normalcy. An explicit formalization of the control scheme is provided for the susceptible--infected--susceptible epidemic model over a network. Its performance and flexibility are demonstrated by means of numerical simulations.