Model predictive control (MPC) has emerged as an effective strategy for water distribution systems (WDSs) management. However, it is hampered by the computational burden for large-scale WDSs due to the combinatorial growth of possible control actions that must be evaluated at each time step. Therefore, a fast computation algorithm to implement MPC in WDSs can be obtained using a move-blocking approach that simplifies control decisions while ensuring solution feasibility. This paper introduces a least-restrictive move-blocking that interpolates the blocked control rate of change, aiming at balancing computational efficiency with operational effectiveness. The proposed control strategy is demonstrated on aggregated WDSs, encompassing multiple hydraulic elements. This implementation is incorporated into a multi-objective optimization framework that concurrently optimizes water level security of the storage tanks, smoothness of the control actions, and cost-effective objectives. A fair comparison between the proposed approach with the non-blocking Economic MPC is provided.
Visual servoing has been gaining popularity in various real-world vision-centric robotic applications. Autonomous robotic grasping often deals with unseen and unstructured environments, and in this task, Visual Servoing has been able to generate improved end-effector control by providing visual feedback. However, existing Servoing-aided grasping methods tend to fail at the task of grasping in dynamic environments i.e. - moving objects.In this paper, we introduce DynGraspVS, a novel Image-based Visual Servoing-aided Grasping approach that models the motion of moving objects in its interaction matrix. Leveraging a single-step rollout strategy, our approach achieves a remarkable increase in success rate, while converging faster and achieving a smoother trajectory, while maintaining precise alignments in six degrees of freedom. By integrating the velocity information into the interaction matrix, our method is able to successfully complete the challenging task of robotic grasping in the case of dynamic objects, while outperforming existing deep Model Predictive Control (MPC) based methods in the PyBullet simulation environment. We test it with a range of objects in the YCB dataset with varying range of shapes, sizes, and material properties. We report various evaluation metrics such as photometric error, success rate, time taken, and trajectory length.
The development of control methods based on data has seen a surge of interest in recent years. When applying data-driven controllers in real-world applications, providing theoretical guarantees for the closed-loop system is of crucial importance to ensure reliable operation. In this review, we provide an overview of data-driven model predictive control (MPC) methods for controlling unknown systems with guarantees on systems-theoretic properties such as stability, robustness, and constraint satisfaction. The considered approaches rely on the Fundamental Lemma from behavioral theory in order to predict input-output trajectories directly from data. We cover various setups, ranging from linear systems and noise-free data to more realistic formulations with noise and nonlinearities, and we provide an overview of different techniques to ensure guarantees for the closed-loop system. Moreover, we discuss avenues for future research that may further improve the theoretical understanding and practical applicability of data-driven MPC.
The tunable approximated explicit model predictive control (MPC) comes with the benefits of real-time tunability without the necessity of solving the optimization problem online. This paper provides a novel self-tunable control policy that does not require any interventions of the control engineer during operation in order to retune the controller subject to the changed working conditions. Based on the current operating conditions, the autonomous tuning parameter scales the control input using linear interpolation between the boundary optimal control actions. The adjustment of the tuning parameter depends on the current reference value, which makes this strategy suitable for reference tracking problems. Furthermore, a novel technique for scaling the tuning parameter is proposed. This extension provides to exploit different ranges of the tuning parameter assigned to specified operating conditions. The self-tunable explicit MPC was implemented on a laboratory heat exchanger with nonlinear and asymmetric behavior. The asymmetric behavior of the plant was compensated by tuning the controller's aggressiveness, as the negative or positive sign of reference change was considered in the tuning procedure. The designed self-tunable controller improved control performance by decreasing sum-of-squared control error, maximal overshoots/ undershoots, and settling time compared to the conventional control strategy based on a single (non-tunable) controller.
This paper addresses the identification of models for offset-free model predictive control (MPC), where LTI models are augmented with (fictitious) uncontrollable integrating modes, called integrating disturbances. The states and disturbances are typically estimated with a Kalman filter. The disturbance estimates effectively provide integral control, so the quality of the disturbance model (and resulting filter) directly influences the control performance. We implement eigenvalue constraints to protect against undesirable filter behavior (unstable or marginally stable modes, high-frequency oscillations). Specifically, we consider the class of linear matrix inequality (LMI) regions for eigenvalue constraints. These LMI regions are open sets by default, so we introduce a barrier function method to create tightened, but closed, eigenvalue constraints. To solve the resulting nonlinear semidefinite program, we approximate it as a nonlinear program using a Cholesky factorization method that exploits known sparsity structures of semidefinite optimization variables and matrix inequalities. The algorithm is applied to real-world data taken from two physical systems: first, a low-cost benchmark temperature microcontroller suitable for classroom laboratories, and second, an industrial-scale chemical reactor at Eastman Chemical's plant in Kingsport, TN.
\Ac{MPC} and \ac{RL} are two powerful control strategies with, arguably, complementary advantages. In this work, we show how actor-critic \ac{RL} techniques can be leveraged to improve the performance of \ac{MPC}. The \ac{RL} critic is used as an approximation of the optimal value function, and an actor roll-out provides an initial guess for primal variables of the \ac{MPC}. A parallel control architecture is proposed where each \ac{MPC} instance is solved twice for different initial guesses. Besides the actor roll-out initialization, a shifted initialization from the previous solution is used. Thereafter, the actor and the critic are again used to approximately evaluate the infinite horizon cost of these trajectories. The control actions from the lowest-cost trajectory are applied to the system at each time step. We establish that the proposed algorithm is guaranteed to outperform the original \ac{RL} policy plus an error term that depends on the accuracy of the critic and decays with the horizon length of the \ac{MPC} formulation. Moreover, we do not require globally optimal solutions for these guarantees to hold. The approach is demonstrated on an illustrative toy example and an \ac{AD} overtaking scenario.
In this paper we describe a new conceptual framework that connects approximate Dynamic Programming (DP), Model Predictive Control (MPC), and Reinforcement Learning (RL). This framework centers around two algorithms, which are designed largely independently of each other and operate in synergy through the powerful mechanism of Newton's method. We call them the off-line training and the on-line play algorithms. The names are borrowed from some of the major successes of RL involving games; primary examples are the recent (2017) AlphaZero program (which plays chess, [SHS17], [SSS17]), and the similarly structured and earlier (1990s) TD-Gammon program (which plays backgammon, [Tes94], [Tes95], [TeG96]). In these game contexts, the off-line training algorithm is the method used to teach the program how to evaluate positions and to generate good moves at any given position, while the on-line play algorithm is the method used to play in real time against human or computer opponents. Significantly, the synergy between off-line training and on-line play also underlies MPC (as well as other major classes of sequential decision problems), and indeed the MPC design architecture is very similar to the one of AlphaZero and TD-Gammon. This conceptual insight provides a vehicle for bridging the cultural gap between RL and MPC, and sheds new light on some fundamental issues in MPC. These include the enhancement of stability properties through rollout, the treatment of uncertainty through the use of certainty equivalence, the resilience of MPC in adaptive control settings that involve changing system parameters, and the insights provided by the superlinear performance bounds implied by Newton's method.
This study presents a nonlinear model predictive control (NMPC) formulation for preview-based traction control, which uses the information on the expected tire-road friction coefficient ahead to enhance the wheel slip control performance, in the context of connected vehicles with V2X features. Proof-of-concept experiments on an electric vehicle prototype highlight the real-time capability of the controller, and the wheel slip control performance improvement brought by the tire-road friction coefficient preview. Finally, an experimentally validated simulation model is used in sensitivity analyses, to evaluate the performance benefit of the preview-based controller for different dynamic characteristics (e.g., time constant and pure time delays) of the electric powertrains.
Path tracking (PT) controllers capable of replicating race driving techniques, such as drifting beyond the limits of handling, have the potential of enhancing active safety in critical conditions. This paper presents a nonlinear model predictive control (NMPC) approach that integrates multiple actuation methods, namely four-wheel-steering, longitudinal tyre force distribution, and direct yaw moment control, to execute drifting when this is beneficial for PT in emergency scenarios. Simulation results of challenging manoeuvres, based on an experimentally validated vehicle model, highlight the substantial PT performance improvements brought by: i) vehicle operation outside the envelope enforced by the current generation of stability controllers; and ii) the integrated control of multiple actuators.
The growing complexity of power system management has led to an increased interest in reinforcement learning (RL). To validate their effectiveness, RL algorithms have to be evaluated across multiple case studies. Case study design is an arduous task requiring the consideration of many aspects, among them the influence of available forecasts and the level of decentralization in the control structure. Furthermore, vanilla RL controllers cannot themselves ensure the satisfaction of system constraints, which makes devising a safeguarding mechanism a necessary task for every case study before deploying the system. To address these shortcomings, we introduce the Python tool CommonPower, the first general framework for the modeling and simulation of power system management tailored towards machine learning. Its modular architecture enables users to focus on specific elements without having to implement a simulation environment. Another unique contribution of CommonPower is the automatic synthesis of model predictive controllers and safeguards. Beyond offering a unified interface for single-agent RL, multi-agent RL, and optimal control, CommonPower includes a training pipeline for machine-learning-based forecasters as well as a flexible mechanism for incorporating feedback of safeguards into the learning updates of RL controllers.
Neural networks are lately more and more often being used in the context of data-driven control, as an approximate model of the true system dynamics. Model Predictive Control (MPC) adopts this practise leading to neural MPC strategies. This raises a question of whether the trained neural network has converged and generalized in a way that the learned model encapsulates an accurate approximation of the true dynamic model of the system, thus making it a reliable choice for model-based control, especially for disturbed and uncertain systems. To tackle that, we propose Dropout MPC, a novel sampling-based ensemble neural MPC algorithm that employs the Monte-Carlo dropout technique on the learned system model. The closed loop is based on an ensemble of predictive controllers, that are used simultaneously at each time-step for trajectory optimization. Each member of the ensemble influences the control input, based on a weighted voting scheme, thus by employing different realizations of the learned system dynamics, neural control becomes more reliable by design. An additional strength of the method is that it offers by design a way to estimate future uncertainty, leading to cautious control. While the method aims in general at uncertain systems with complex dynamics, where models derived from first principles are hard to infer, to showcase the application we utilize data gathered in the laboratory from a real mobile manipulator and employ the proposed algorithm for the navigation of the robot in simulation.
Distributed model predictive control (DMPC) has attracted extensive attention as it can explicitly handle system constraints and achieve optimal control in a decentralized manner. However, the deployment of DMPC strategies generally requires the sharing of sensitive data among subsystems, which may violate the privacy of participating systems. In this paper, we propose a differentially-private DMPC algorithm for linear discrete-time systems subject to coupled global constraints. Specifically, we first show that a conventional distributed dual gradient algorithm can be used to address the considered DMPC problem but cannot provide strong privacy preservation. Then, to protect privacy against the eavesdropper, we incorporate a differential-privacy noise injection mechanism into the DMPC framework and prove that the resulting distributed optimization algorithm can ensure both provable convergence to a global optimal solution and rigorous $\epsilon$-differential privacy. In addition, an implementation strategy of the DMPC is designed such that the recursive feasibility and stability of the closed-loop system are guaranteed. Simulation results are provided to demonstrate the effectiveness of the developed approach.
Nonlinear Programs (NLPs) are prevalent in optimization-based control of nonlinear systems. Solving general NLPs is computationally expensive, necessitating the development of fast hardware or tractable suboptimal approximations. This paper investigates the sensitivity of the solutions of NLPs with polytopic constraints when the nonlinear continuous objective function is approximated by a PieceWise-Affine (PWA) counterpart. By leveraging perturbation analysis using a convex modulus, we derive guaranteed bounds on the distance between the optimal solution of the original polytopically-constrained NLP and that of its approximated formulation. Our approach aids in determining criteria for achieving desired solution bounds. Two case studies on the Eggholder function and nonlinear model predictive control of an inverted pendulum demonstrate the theoretical results.
With neural networks being used to control safety-critical systems, they increasingly have to be both accurate (in the sense of matching inputs to outputs) and robust. However, these two properties are often at odds with each other and a trade-off has to be navigated. To address this issue, this paper proposes a method to generate an approximation of a neural network which is certifiably more robust. Crucially, the method is fully convex and posed as a semi-definite programme. An application to robustifying model predictive control is used to demonstrate the results. The aim of this work is to introduce a method to navigate the neural network robustness/accuracy trade-off.
Model predictive control can optimally deal with nonlinear systems under consideration of constraints. The control performance depends on the model accuracy and the prediction horizon. Recent advances propose to use reinforcement learning applied to a parameterized model predictive controller to recover the optimal control performance even if an imperfect model or short prediction horizons are used. However, common reinforcement learning algorithms rely on first order updates, which only have a linear convergence rate and hence need an excessive amount of dynamic data. Higher order updates are typically intractable if the policy is approximated with neural networks due to the large number of parameters. In this work, we use a parameterized model predictive controller as policy, and leverage the small amount of necessary parameters to propose a trust-region constrained Quasi-Newton training algorithm for policy optimization with a superlinear convergence rate. We show that the required second order derivative information can be calculated by the solution of a linear system of equations. A simulation study illustrates that the proposed training algorithm outperforms other algorithms in terms of data efficiency and accuracy.
In this project, we attempt to optimize a landing trajectory of a rocket. The goal is to minimize the total fuel consumption during the landing process using different techniques. Once the optimal and feasible trajectory is generated using batch approach, we attempt to follow the path using a Model Predictive Control (MPC) based algorithm, called Trajectory Optimizing Path following Estimation from Demonstration (TOPED), in order to generalize to similar initial states and models, where we introduce a novel cost function for the MPC to solve. We further show that TOPED can follow a demonstration trajectory well in practice under model mismatch and different initial states.
Model-based Reinforcement Learning (MBRL) has shown many desirable properties for intelligent control tasks. However, satisfying safety and stability constraints during training and rollout remains an open question. We propose a new Model-based RL framework to enable efficient policy learning with unknown dynamics based on learning model predictive control (LMPC) framework with mathematically provable guarantees of stability. We introduce and explore a novel method for adding safety constraints for model-based RL during training and policy learning. The new stability-augmented framework consists of a neural-network-based learner that learns to construct a Lyapunov function, and a model-based RL agent to consistently complete the tasks while satisfying user-specified constraints given only sub-optimal demonstrations and sparse-cost feedback. We demonstrate the capability of the proposed framework through simulated experiments.
As the number of Persons with Disabilities (PWD), particularly those with one or more physical impairments, increases, there is an increasing demand for assistive robotic technologies that can support independent mobility in the built environment and reduce the burden on caregivers. Current assistive mobility platforms (e.g., robotic wheelchairs) often fail to incorporate user preferences and control, leading to reduced trust and efficiency. Existing shared control algorithms do not allow the incorporation of the user control preferences inside the navigation framework or the path planning algorithm. In addition, existing dynamic local planner algorithms for robotic wheelchairs do not take into account the social spaces of people, potentially leading such platforms to infringe upon these areas and cause discomfort. To address these concerns, this work introduces a novel socially-aware shared autonomy-based navigation system for assistive mobile robotic platforms. Our navigation framework comprises a Global Planner and a Local Planner. To implement the Global Planner, the proposed approach introduces a novel User Preference Field (UPF) theory within its global planning framework, explicitly acknowledging user preferences to adeptly navigate away from congested areas. For the Local Planner, we propose a Socially-aware Shared Control-based Model Predictive Control with Dynamic Control Barrier Function (SS-MPC-DCBF) to adjust movements in real-time, integrating user preferences for safer, more autonomous navigation. Evaluation results show that our Global Planner aligns closely with user preferences compared to baselines, and our Local Planner demonstrates enhanced safety and efficiency in dynamic and static scenarios. This integrated approach fosters trust and autonomy, crucial for the acceptance of assistive mobility technologies in the built environment.
This paper presents a data-driven Model Predictive Control (MPC) for energy-efficient urban road driving for connected, automated vehicles. The proposed MPC aims to minimize total energy consumption by controlling the vehicle's longitudinal motion on roads with traffic lights and front vehicles. Its terminal cost function and terminal constraints are learned from data, which consists of the closed-loop state and input trajectories. The terminal cost function represents the remaining energy-to-spend starting from a given terminal state. The terminal constraints are designed to ensure that the controlled vehicle timely crosses the upcoming traffic light, adheres to traffic laws, and accounts for the front vehicles. We validate the effectiveness of our method through both simulations and vehicle-in-the-loop experiments, demonstrating 19% improvement in average energy efficiency compared to conventional approaches that involve solving a long-horizon optimal control problem for speed planning and employing a separate controller for speed tracking.
Safe, socially compliant, and efficient navigation of low-speed autonomous vehicles (AVs) in pedestrian-rich environments necessitates considering pedestrians' future positions and interactions with the vehicle and others. Despite the inevitable uncertainties associated with pedestrians' predicted trajectories due to their unobserved states (e.g., intent), existing deep reinforcement learning (DRL) algorithms for crowd navigation often neglect these uncertainties when using predicted trajectories to guide policy learning. This omission limits the usability of predictions when diverging from ground truth. This work introduces an integrated prediction and planning approach that incorporates the uncertainties of predicted pedestrian states in the training of a model-free DRL algorithm. A novel reward function encourages the AV to respect pedestrians' personal space, decrease speed during close approaches, and minimize the collision probability with their predicted paths. Unlike previous DRL methods, our model, designed for AV operation in crowded spaces, is trained in a novel simulation environment that reflects realistic pedestrian behaviour in a shared space with vehicles. Results show a 40% decrease in collision rate and a 15% increase in minimum distance to pedestrians compared to the state of the art model that does not account for prediction uncertainty. Additionally, the approach outperforms model predictive control methods that incorporate the same prediction uncertainties in terms of both performance and computational time, while producing trajectories closer to human drivers in similar scenarios.