A trust-aware safe control system for autonomous navigation in the presence of humans, specifically pedestrians, is presented. The system combines model predictive control (MPC) with control barrier functions (CBFs) and trust estimation to ensure safe and reliable navigation in complex environments. Pedestrian trust values are computed based on features, extracted from camera sensor images, such as mutual eye contact and smartphone usage. These trust values are integrated into the MPC controller's CBF constraints, allowing the autonomous vehicle to make informed decisions considering pedestrian behavior. Simulations conducted in the CARLA driving simulator demonstrate the feasibility and effectiveness of the proposed system, showcasing more conservative behaviour around inattentive pedestrians and vice versa. The results highlight the practicality of the system in real-world applications, providing a promising approach to enhance the safety and reliability of autonomous navigation systems, especially self-driving vehicles.
We present the design and \textit{in-silico} evaluation of a closed-loop insulin delivery algorithm to treat type 1 diabetes (T1D) consisting in a data-driven multi-step-ahead blood glucose (BG) predictor integrated into a Linear Time-Varying (LTV) Model Predictive Control (MPC) framework. Instead of identifying an open-loop model of the glucoregulatory system from available data, we propose to directly fit the entire BG prediction over a predefined prediction horizon to be used in the MPC, as a nonlinear function of past input-ouput data and an affine function of future insulin control inputs. For the nonlinear part, a Long Short-Term Memory (LSTM) network is proposed, while for the affine component a linear regression model is chosen. To assess benefits and drawbacks when compared to a traditional linear MPC based on an auto-regressive with exogenous (ARX) input model identified from data, we evaluated the proposed LSTM-MPC controller in three simulation scenarios: a nominal case with 3 meals per day, a random meal disturbances case where meals were generated with a recently published meal generator, and a case with 25$\%$ decrease in the insulin sensitivity. Further, in all the scenarios, no feedforward meal bolus was administered. For the more challenging random meal generation scenario, the mean $\pm$ standard deviation percent time in the range 70-180 [mg/dL] was 74.99 $\pm$ 7.09 vs. 54.15 $\pm$ 14.89, the mean $\pm$ standard deviation percent time in the tighter range 70-140 [mg/dL] was 47.78$\pm$8.55 vs. 34.62 $\pm$9.04, while the mean $\pm$ standard deviation percent time in sever hypoglycemia, i.e., $<$ 54 [mg/dl] was 1.00$\pm$3.18 vs. 9.45$\pm$11.71, for our proposed LSTM-MPC controller and the traditional ARX-MPC, respectively. Our approach provided accurate predictions of future glucose concentrations and good closed-loop performances of the overall MPC controller.
This paper considers controlling automated vehicles (AVs) to form a platoon with human-driven vehicles (HVs) under consideration of unknown HV model parameters and propulsion time constants. The proposed design is a data-driven dual-loop control strategy for the ego AVs, where the inner loop controller ensures platoon stability and the outer loop controller keeps a safe inter-vehicular spacing under control input limits. The inner loop controller is a constant-gain state feedback controller solved from a semidefinite program (SDP) using the online collected data of platooning errors. The outer loop is a model predictive control (MPC) that embeds a data-driven internal model to predict the future platooning error evolution. The proposed design is evaluated on a mixed platoon with a representative aggressive reference velocity profile, the SFTP-US06 Drive Cycle. The results confirm efficacy of the design and its advantages over the existing single loop data-driven MPC in terms of platoon stability and computational cost.
This work proposes a finite-horizon optimal control strategy to solve the tracking problem while providing avoidance features to the closed-loop system. Inspired by the set-point tracking model predictive control (MPC) framework, the central idea of including artificial variables into the optimal control problem is considered. This approach allows us to add avoidance features into the set-point tracking MPC strategy without losing the properties of an enlarged domain of attraction and feasibility insurances in the face of any changing reference. Besides, the artificial variables are considered together with an avoidance cost functional to establish the basis of the strategy, maintaining the recursive feasibility property in the presence of a previously unknown number of regions to be avoided. It is shown that the closed-loop system is recursively feasible and input-to-state-stable under the mild assumption that the avoidance cost is uniformly bounded over time. Finally, two numerical examples illustrate the controller behavior.
In this paper, we propose a parallel shooting algorithm for solving nonlinear model predictive control problems using sequential quadratic programming. This algorithm is built on a two-phase approach where we first test and assess sequential convergence over many initial trajectories in parallel. However, if none converge, the algorithm starts varying the Newton step size in parallel instead. Through this parallel shooting approach, it is expected that the number of iterations to converge to an optimal solution can be decreased. Furthermore, the algorithm can be further expanded and accelerated by implementing it on GPUs. We illustrate the effectiveness of the proposed Parallel Shooting Sequential Quadratic Programming (PS-SQP) method in some benchmark examples for nonlinear model predictive control. The developed PS-SQP parallel solver converges faster on average and especially when significant nonlinear behaviour is excited in the NMPC horizon.
Learning-based optimal control algorithms control unknown systems using past trajectory data and a learned model of the system dynamics. These controllers use either a linear approximation of the learned dynamics, trading performance for faster computation, or nonlinear optimization methods, which typically perform better but can limit real-time applicability. In this work, we present a novel nonlinear controller that exploits differential flatness to achieve similar performance to state-of-the-art learning-based controllers but with significantly less computational effort. Differential flatness is a property of dynamical systems whereby nonlinear systems can be exactly linearized through a nonlinear input mapping. Here, the nonlinear transformation is learned as a Gaussian process and is used in a safety filter that guarantees, with high probability, stability as well as input and flat state constraint satisfaction. This safety filter is then used to refine inputs from a flat model predictive controller to perform constrained nonlinear learning-based optimal control through two successive convex optimizations. We compare our method to state-of-the-art learning-based control strategies and achieve similar performance, but with significantly better computational efficiency, while also respecting flat state and input constraints, and guaranteeing stability.
In this paper we propose a robust Model Predictive Control where a Gated Recurrent Unit network model is used to learn the input-output dynamic of the system under control. Robust satisfaction of input and output constraints and recursive feasibility in presence of model uncertainties are achieved using a constraint tightening approach. Moreover, new terminal cost and terminal set are introduced in the Model Predictive Control formulation to guarantee Input-to-State Stability of the closed loop system with respect to the uncertainty term.
Stable gait generation is a crucial problem for legged robot locomotion as this impacts other critical performance factors such as, e.g. mobility over an uneven terrain and power consumption. Gait generation stability results from the efficient control of the interaction between the legged robot's body and the environment where it moves. Here, we study how this can be achieved by a combination of model-predictive and predictive reinforcement learning controllers. Model-predictive control (MPC) is a well-established method that does not utilize any online learning (except for some adaptive variations) as it provides a convenient interface for state constraints management. Reinforcement learning (RL), in contrast, relies on adaptation based on pure experience. In its bare-bone variants, RL is not always suitable for robots due to their high complexity and expensive simulation/experimentation. In this work, we combine both control methods to address the quadrupedal robot stable gate generation problem. The hybrid approach that we develop and apply uses a cost roll-out algorithm with a tail cost in the form of a Q-function modeled by a neural network; this allows to alleviate the computational complexity, which grows exponentially with the prediction horizon in a purely MPC approach. We demonstrate that our RL gait controller achieves stable locomotion at short horizons, where a nominal MP controller fails. Further, our controller is capable of live operation, meaning that it does not require previous training. Our results suggest that the hybridization of MPC with RL, as presented here, is beneficial to achieve a good balance between online control capabilities and computational complexity.
In their seminal 1990 paper, Wasserman and Kadane establish an upper bound for the Bayes' posterior probability of a measurable set $A$, when the prior lies in a class of probability measures $\mathcal{P}$ and the likelihood is precise. They also give a sufficient condition for such upper bound to hold with equality. In this paper, we introduce a generalization of their result by additionally addressing uncertainty related to the likelihood. We give an upper bound for the posterior probability when both the prior and the likelihood belong to a set of probabilities. Furthermore, we give a sufficient condition for this upper bound to become an equality. This result is interesting on its own, and has the potential of being applied to various fields of engineering (e.g. model predictive control), machine learning, and artificial intelligence.
Objective: The artificial pancreas (AP) has shown promising potential in achieving closed-loop glucose control for individuals with type 1 diabetes mellitus (T1DM). However, designing an effective control policy for the AP remains challenging due to the complex physiological processes, delayed insulin response, and inaccurate glucose measurements. While model predictive control (MPC) offers safety and stability through the dynamic model and safety constraints, it lacks individualization and is adversely affected by unannounced meals. Conversely, deep reinforcement learning (DRL) provides personalized and adaptive strategies but faces challenges with distribution shifts and substantial data requirements. Methods: We propose a hybrid control policy for the artificial pancreas (HyCPAP) to address the above challenges. HyCPAP combines an MPC policy with an ensemble DRL policy, leveraging the strengths of both policies while compensating for their respective limitations. To facilitate faster deployment of AP systems in real-world settings, we further incorporate meta-learning techniques into HyCPAP, leveraging previous experience and patient-shared knowledge to enable fast adaptation to new patients with limited available data. Results: We conduct extensive experiments using the FDA-accepted UVA/Padova T1DM simulator across three scenarios. Our approaches achieve the highest percentage of time spent in the desired euglycemic range and the lowest occurrences of hypoglycemia. Conclusion: The results clearly demonstrate the superiority of our methods for closed-loop glucose management in individuals with T1DM. Significance: The study presents novel control policies for AP systems, affirming the great potential of proposed methods for efficient closed-loop glucose control.
In this study, we are concerned with autonomous driving missions when a static obstacle blocks a given reference trajectory. To provide a realistic control design, we employ a model predictive control (MPC) utilizing nonlinear state-space dynamic models of a car with linear tire forces, allowing for optimal path planning and tracking to overtake the obstacle. We provide solutions with two different methodologies. Firstly, we solve a nonlinear MPC (NMPC) problem with a nonlinear optimization framework, capable of considering the nonlinear constraints. Secondly, by introducing scheduling signals, we embed the nonlinear dynamics in a linear parameter varying (LPV) representation with adaptive linear constraints for realizing the nonlinear constraints associated with the obstacle. Consequently, an LPVMPC optimization problem can be solved efficiently as a quadratic programming (QP) that constitutes the main novelty of this work. We test the two methods for a challenging obstacle avoidance task and provide qualitative comparisons. The LPVMPC shows a significant reduction in terms of the computational burden at the expense of a slight loss of performance.
A novel human-machine interactive training method for dynamic load change operation in air separation processes (ASPs) is proposed. A shadow operator (SO) is developed in this method to train ASP operators through industrial model predictive control (IMPC) and Bloom's taxonomy. First, a nonlinear two-layer IMPC machine algorithm is developed for dynamic load change operation. The IMPC uses a linear parameter varying prediction model and an iterative multi-step linearization algorithm to compute accurate control decisions. Second, a hierarchical human-machine cooperation model is established to improve the effectiveness of operation training. The model is inspired by an educational psychology framework (Bloom's taxonomy) and assists ASP operators in enhancing their dynamic operational skills. Finally, five dynamic training modes of the SO are designed based on the IMPC algorithm and the human-machine cooperation model. The practical application results demonstrate that the SO improves the effectiveness of skill acquisition for novice operators and the safety of dynamic operations.
This article provides an overview of model predictive control (MPC) frameworks for dynamic operation of nonlinear constrained systems. Dynamic operation is often an integral part of the control objective, ranging from tracking of reference signals to the general economic operation of a plant under online changing time-varying operating conditions. We focus on the particular challenges that arise when dealing with such more general control goals and present methods that have emerged in the literature to address these issues. The goal of this article is to present an overview of the state-of-the-art techniques, providing a diverse toolkit to apply and further develop MPC formulations that can handle the challenges intrinsic to dynamic operation. We also critically assess the applicability of the different research directions, discussing limitations and opportunities for further research.
This paper describes an intelligent management algorithm for an aggregate of domestic electric water heaters called to provide a demand response service. This algorithm is developed using Model Predictive Control. The model of the entire aggregate is dynamically identified using a recursive polynomial model estimation technique. This allows the control to be adaptive, i.e., able to adjust its decisions to the system characteristics, which vary over time due to the daily distribution of users hot water consumption. To answer the demand response requirements, aggregated power variations are realized by modifying the temperature set-points of the water heaters without compromising the users comfort. The developed approach allows tracking a regulation signal and mitigating the so-called rebound, i.e., the recovery of energy needed by the aggregate at the end of the service to return to the baseline thermal state. Analyses in a simulation environment allow the validation of the potentialities of the proposed method.
There are two major challenges for scaling up robot navigation around dynamic obstacles: the complex interaction dynamics of the obstacles can be hard to model analytically, and the complexity of planning and control grows exponentially in the number of obstacles. Data-driven and learning-based methods are thus particularly valuable in this context. However, data-driven methods are sensitive to distribution drift, making it hard to train and generalize learned models across different obstacle densities. We propose a novel method for compositional learning of Sequential Neural Control Barrier models (SNCBFs) to achieve scalability. Our approach exploits an important observation: the spatial interaction patterns of multiple dynamic obstacles can be decomposed and predicted through temporal sequences of states for each obstacle. Through decomposition, we can generalize control policies trained only with a small number of obstacles, to environments where the obstacle density can be 100x higher. We demonstrate the benefits of the proposed methods in improving dynamic collision avoidance in comparison with existing methods including potential fields, end-to-end reinforcement learning, and model-predictive control. We also perform hardware experiments and show the practical effectiveness of the approach in the supplementary video.
This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is applied. The idea is to fix the disturbance ellipsoids within one optimization iteration and solve the problem repeatedly with updated disturbance ellipsoid trajectories. The zero-order approach is suboptimal in general. However, we show that it does not impair convergence to the reference trajectory in the absence of obstacles. The experiments on an industrial mobile robot prototype demonstrate the performance of the controller.
Despite great successes, model predictive control (MPC) relies on an accurate dynamical model and requires high onboard computational power, impeding its wider adoption in engineering systems, especially for nonlinear real-time systems with limited computation power. These shortcomings of MPC motivate this work to make such a control framework more practically viable for real-world applications. Specifically, to remove the required accurate dynamical model and reduce the computational cost for nonlinear MPC (NMPC), this paper develops a unified online data-driven predictive control pipeline to efficiently control a system with guaranteed safety without incurring large computational complexity. The new aspect of this idea is learning not only the real system but also the control policy, which results in a reasonable computational cost for the data-driven predictive controllers. More specifically, we first develop a spatial temporal filter (STF)-based concurrent learning scheme to systematically identify system dynamics for general nonlinear systems. We then develop a robust control barrier function (RCBF) for safety guarantees in the presence of model uncertainties and learn the RCBF-based NMPC policy. Furthermore, to mitigate the performance degradation due to the existing model uncertainties, we propose an online policy correction scheme through perturbation analysis and design of an ancillary feedback controller. Finally, extensive simulations on two applications, cart-inverted pendulum and automotive powertrain control, are performed to demonstrate the efficacy of the proposed framework, which shows comparable performance with much lower computational cost in comparison with several benchmark algorithms.
We present a novel data-driven Model Predictive Control (MPC) algorithm for nonlinear systems. The method is based on recent extensions of behavioural theory and Willem's Fundamental Lemma for nonlinear systems by the means of adequate Input-Output (IO) quasi-Linear Parameter Varying (qLPV) embeddings. Thus, the MPC is formulated to ensure regulation and IO constraints satisfaction, based only on measured datasets of sufficient length (and under persistent excitation). Instead of requiring the availability of the scheduling trajectories (as in recent papers), we consider an estimate of the function that maps the qLPV realisation. Specifically, we use an extrapolation procedure in order to generate the future scheduling trajectories, at each sample, which is shown to be convergent and generated bounded errors. Accordingly, we discuss the issues of closed-loop IO stability and recursive feasibility certificates of the method. The algorithm is tested and discussed with the aid of a numerical application.
Dynamics models learned from visual observations have shown to be effective in various robotic manipulation tasks. One of the key questions for learning such dynamics models is what scene representation to use. Prior works typically assume representation at a fixed dimension or resolution, which may be inefficient for simple tasks and ineffective for more complicated tasks. In this work, we investigate how to learn dynamic and adaptive representations at different levels of abstraction to achieve the optimal trade-off between efficiency and effectiveness. Specifically, we construct dynamic-resolution particle representations of the environment and learn a unified dynamics model using graph neural networks (GNNs) that allows continuous selection of the abstraction level. During test time, the agent can adaptively determine the optimal resolution at each model-predictive control (MPC) step. We evaluate our method in object pile manipulation, a task we commonly encounter in cooking, agriculture, manufacturing, and pharmaceutical applications. Through comprehensive evaluations both in the simulation and the real world, we show that our method achieves significantly better performance than state-of-the-art fixed-resolution baselines at the gathering, sorting, and redistribution of granular object piles made with various instances like coffee beans, almonds, corn, etc.
A hierarchical architecture for the optimal management of an ensemble of steam generators is presented. The subsystems are coordinated by a multilayer scheme for jointly sustaining a common load. The high level optimizes the load allocation and the generator schedule, considering activation dynamics by a hybrid model. At the medium level, a robust tube-based model predictive control (MPC) tracks a time-varying demand using a centralized--but aggregate--model, whose order does not scale with the number of subsystems. A nonlinear optimization, at medium level, addresses MPC infeasibility due to abrupt changes of ensemble configuration. Low-level decentralized controllers stabilize the generators. This control scheme enables the dynamical modification of the ensemble configuration and plug and play operations. Simulations demonstrate the approach potentialities.