In a dynamic distribution market environment, residential prosumers with solar power generation and battery energy storage devices can flexibly interact with the power grid via power exchange. Providing a schedule of this bidirectional power dispatch can facilitate the operational planning for the grid operator and bring additional benefits to the prosumers with some economic incentives. However, the major obstacle to achieving this win-win situation is the difficulty in 1) predicting the nonlinear behaviors of battery degradation under unknown operating conditions and 2) addressing the highly uncertain generation/load patterns, in a computationally viable way. This paper thus establishes a robust short-term dispatch framework for residential prosumers equipped with rooftop solar photovoltaic panels and household batteries. The objective is to achieve the minimum-cost operation under the dynamic distribution energy market environment with stipulated dispatch rules. A general nonlinear optimization problem is formulated, taking into consideration the operating costs due to electricity trading, battery degradation, and various operating constraints. The optimization problem is solved in real-time using a proposed ensemble nonlinear model predictive control-based economic dispatch strategy, where the uncertainty in the forecast has been addressed adequately albeit with limited local data. The effectiveness of the proposed algorithm has been validated using real-world prosumer datasets.
The problem of coverage control, i.e., of coordinating multiple agents to optimally cover an area, arises in various applications. However, coverage applications face two major challenges: (1) dealing with nonlinear dynamics while respecting system and safety critical constraints, and (2) performing the task in an initially unknown environment. We solve the coverage problem by using a hierarchical framework, in which references are calculated at a central server and passed to the agents' local model predictive control (MPC) tracking schemes. Furthermore, to ensure that the environment is actively explored by the agents a probabilistic exploration-exploitation trade-off is deployed. In addition, we derive a control framework that avoids the hierarchical structure by integrating the reference optimization in the MPC formulation. Active learning is then performed drawing inspiration from Upper Confidence Bound (UCB) approaches. For all developed control architectures, we guarantee closed-loop constraint satisfaction and convergence to an optimal configuration. Furthermore, all methods are tested and compared on hardware using a miniature car platform.
Hybrid systems are prevalent in robotics. However, ensuring the stability of hybrid systems is challenging due to sophisticated continuous and discrete dynamics. A system with all its system modes stable can still be unstable. Hence special treatments are required at mode switchings to stabilize the system. In this work, we propose a hierarchical, neural network (NN)-based method to control general hybrid systems. For each system mode, we first learn an NN Lyapunov function and an NN controller to ensure the states within the region of attraction (RoA) can be stabilized. Then an RoA NN estimator is learned across different modes. Upon mode switching, we propose a differentiable planner to ensure the states after switching can land in next mode's RoA, hence stabilizing the hybrid system. We provide novel theoretical stability guarantees and conduct experiments in car tracking control, pogobot navigation, and bipedal walker locomotion. Our method only requires 0.25X of the training time as needed by other learning-based methods. With low running time (10-50X faster than model predictive control (MPC)), our controller achieves a higher stability/success rate over other baselines such as MPC, reinforcement learning (RL), common Lyapunov methods (CLF), linear quadratic regulator (LQR), quadratic programming (QP) and Hamilton-Jacobian-based methods (HJB). The project page is on https://mit-realm.github.io/hybrid-clf.
In this paper, a modified robust model predictive control scheme is proposed for linear parametric variable (LPV) and hybrid systems based on a quasi-min-max algorithm. Using a new cost function resulted in reduced unwanted disturbances during switching. In addition, the effects of uncertainties are reduced in the prediction dynamics, and robust stability of piecewise switching LPV systems subjected to linear matrix inequality (LMI) input constraints are guaranteed. Simulation results demonstrate the performance of the proposed controller compared to prior research.
This paper considers the problem of data generation for MPC policy approximation. Learning an approximate MPC policy from expert demonstrations requires a large data set consisting of optimal state-action pairs, sampled across the feasible state space. Yet, the key challenge of efficiently generating the training samples has not been studied widely. Recently, a sensitivity-based data augmentation framework for MPC policy approximation was proposed, where the parametric sensitivities are exploited to cheaply generate several additional samples from a single offline MPC computation. The error due to augmenting the training data set with inexact samples was shown to increase with the size of the neighborhood around each sample used for data augmentation. Building upon this work, this letter paper presents an improved data augmentation scheme based on predictor-corrector steps that enforces a user-defined level of accuracy, and shows that the error bound of the augmented samples are independent of the size of the neighborhood used for data augmentation.
In this paper, we propose a novel approach for computing robust backward reachable sets from noisy data for unknown constrained linear systems subject to bounded disturbances. In particular, we develop an algorithm for obtaining zonotopic inner approximations that can be used for control purposes. It is shown that such sets, if built on an extended space including states and inputs, can be used to embed the system's one-step evolution in the computed extended regions. Such a result is then exploited to build a set-theoretic model predictive controller that, offline, builds a recursive family of robust data-driven reachable sets, and online, computes recursively admissible control actions without explicitly resorting to either a model of the system or the available data.The validity of the proposed data-driven solution is verified by means of a numerical simulation and its performance is contrasted with the model-based counterpart.
Large-scale self-supervised models have recently revolutionized our ability to perform a variety of tasks within the vision and language domains. However, using such models for autonomous systems is challenging because of safety requirements: besides executing correct actions, an autonomous agent must also avoid the high cost and potentially fatal critical mistakes. Traditionally, self-supervised training mainly focuses on imitating previously observed behaviors, and the training demonstrations carry no notion of which behaviors should be explicitly avoided. In this work, we propose Control Barrier Transformer (ConBaT), an approach that learns safe behaviors from demonstrations in a self-supervised fashion. ConBaT is inspired by the concept of control barrier functions in control theory and uses a causal transformer that learns to predict safe robot actions autoregressively using a critic that requires minimal safety data labeling. During deployment, we employ a lightweight online optimization to find actions that ensure future states lie within the learned safe set. We apply our approach to different simulated control tasks and show that our method results in safer control policies compared to other classical and learning-based methods such as imitation learning, reinforcement learning, and model predictive control.
Predictive control, which is based on a model of the system to compute the applied input optimizing the future system behavior, is by now widely used. If the nominal models are not given or are very uncertain, data-driven model predictive control approaches can be employed, where the system model or input is directly obtained from past measured trajectories. Using a data informativity framework and Finsler's lemma, we propose a data-driven robust linear matrix inequality-based model predictive control scheme that considers input and state constraints. Using these data, we formulate the problem as a semi-definite optimization problem, whose solution provides the matrix gain for the linear feedback, while the decisive variables are independent of the length of the measurement data. The designed controller stabilizes the closed-loop system asymptotically and guarantees constraint satisfaction. Numerical examples are conducted to illustrate the method.
We propose a model predictive control approach for autonomous vehicles that exploits learned Gaussian processes for predicting human driving behavior. The proposed approach employs the uncertainty about the GP's prediction to achieve safety. A multi-mode predictive control approach considers the possible intentions of the human drivers. While the intentions are represented by different Gaussian processes, their probabilities foreseen in the observed behaviors are determined by a suitable online classification. Intentions below a certain probability threshold are neglected to improve performance. The proposed multi-mode model predictive control approach with Gaussian process regression support enables repeated feasibility and probabilistic constraint satisfaction with high probability. The approach is underlined in simulation, considering real-world measurements for training the Gaussian processes.
We study the problem of model predictive control (MPC) for the fish schooling model proposed by Gautrais et al. (Annales Zoologici Fennici, 2008). The high nonlinearity of the model attributed to its attraction/alignment/repulsion law suggests the need to use MPC for controlling the fish schooling's motion. However, for large schools, the hybrid nature of the law can make it numerically demanding to perform finite-horizon optimizations in MPC. Therefore, this paper proposes reducing the fish schooling model for numerically efficient MPC; the reduction is based on using the weighted average of the directions of individual fish in the school. We analytically show how using the normalized eigenvector centrality of the alignment-interaction network can yield a better reduction by comparing reduction errors. We confirm this finding on the weight and numerical efficiency of the MPC with the reduced-order model by numerical simulations. The proposed reduction allows us to control a school with up to 500 individuals. Further, we confirm that reduction with the normalized eigenvector centrality allows us to improve the control accuracy by factor of five when compared to that using constant weights.
Model-based RL/control have gained significant traction in robotics. Yet, these approaches often remain data-inefficient and lack the explainability of hand-engineered solutions. This makes them difficult to debug/integrate in safety-critical settings. However, in many systems, prior knowledge of environment kinematics/dynamics is available. Incorporating such priors can help address the aforementioned problems by reducing problem complexity and the need for exploration, while also facilitating the expression of the decisions taken by the agent in terms of physically meaningful entities. Our aim with this paper is to illustrate and support this point of view via a case-study. We model a payload manipulation problem based on a real robotic system, and show that leveraging prior knowledge about the dynamics of the environment in an MPC framework can lead to improvements in explainability, safety and data-efficiency, leading to satisfying generalization properties with less data.
In this paper, a model predictive mixed integer control method for BYD Qin Plus DM-i (Dual Model intelligent) plug-in hybrid electric vehicle (PHEV) is proposed for co-optimization to reduce fuel consumption during car following. First, the adaptive cruise control (ACC) model for energy-saving driving is established. Then, a control-oriented energy management strategy (EMS) model considering the clutch engagement and disengagement is constructed. Finally, the co-optimization structure by integrating ACC model and EMS model is created and is converted to the mixed integer nonlinear programming (MINLP). The results show that this modeling method can be applied to EMS based on the model predictive control (MPC) framework and verify that co-optimization can achieve a 5.1$\%$ reduction in fuel consumption compared to sequential optimization with the guarantee of ACC performance.
This paper presents a deep learning based model predictive control algorithm for control affine nonlinear discrete time systems with matched and bounded state-dependent uncertainties of unknown structure. Since the structure of uncertainties is not known, a deep neural network (DNN) is employed to approximate the disturbances. In order to avoid any unwanted behavior during the learning phase, a tube based model predictive controller is employed, which ensures satisfaction of constraints and input-to-state stability of the closed-loop states.
Quadruped animal locomotion emerges from the interactions between the spinal central pattern generator (CPG), sensory feedback, and supraspinal drive signals from the brain. Computational models of CPGs have been widely used for investigating the spinal cord contribution to animal locomotion control in computational neuroscience and in bio-inspired robotics. However, the contribution of supraspinal drive to anticipatory behavior, i.e. motor behavior that involves planning ahead of time (e.g. of footstep placements), is not yet properly understood. In particular, it is not clear whether the brain modulates CPG activity and/or directly modulates muscle activity (hence bypassing the CPG) for accurate foot placements. In this paper, we investigate the interaction of supraspinal drive and a CPG in an anticipatory locomotion scenario that involves stepping over gaps. By employing deep reinforcement learning (DRL), we train a neural network policy that replicates the supraspinal drive behavior. This policy can either modulate the CPG dynamics, or directly change actuation signals to bypass the CPG dynamics. Our results indicate that the direct supraspinal contribution to the actuation signal is a key component for a high gap crossing success rate. However, the CPG dynamics in the spinal cord are beneficial for gait smoothness and energy efficiency. Moreover, our investigation shows that sensing the front feet distances to the gap is the most important and sufficient sensory information for learning gap crossing. Our results support the biological hypothesis that cats and horses mainly control the front legs for obstacle avoidance, and that hind limbs follow an internal memory based on the front limbs' information. Our method enables the quadruped robot to cross gaps of up to 20 cm (50% of body-length) without any explicit dynamics modeling or Model Predictive Control (MPC).
Many real-world offline reinforcement learning (RL) problems involve continuous-time environments with delays. Such environments are characterized by two distinctive features: firstly, the state x(t) is observed at irregular time intervals, and secondly, the current action a(t) only affects the future state x(t + g) with an unknown delay g > 0. A prime example of such an environment is satellite control where the communication link between earth and a satellite causes irregular observations and delays. Existing offline RL algorithms have achieved success in environments with irregularly observed states in time or known delays. However, environments involving both irregular observations in time and unknown delays remains an open and challenging problem. To this end, we propose Neural Laplace Control, a continuous-time model-based offline RL method that combines a Neural Laplace dynamics model with a model predictive control (MPC) planner--and is able to learn from an offline dataset sampled with irregular time intervals from an environment that has a inherent unknown constant delay. We show experimentally on continuous-time delayed environments it is able to achieve near expert policy performance.
In this paper, we tackle the problem of Unmanned Aerial (UA V) path planning in complex and uncertain environments by designing a Model Predictive Control (MPC), based on a Long-Short-Term Memory (LSTM) network integrated into the Deep Deterministic Policy Gradient algorithm. In the proposed solution, LSTM-MPC operates as a deterministic policy within the DDPG network, and it leverages a predicting pool to store predicted future states and actions for improved robustness and efficiency. The use of the predicting pool also enables the initialization of the critic network, leading to improved convergence speed and reduced failure rate compared to traditional reinforcement learning and deep reinforcement learning methods. The effectiveness of the proposed solution is evaluated by numerical simulations.
This paper presents a novel set-based model predictive control for tracking, which provides the largest domain of attraction, even with the minimal predictive/control horizon. The formulation - which consists of a single optimization problem - shows a dual behavior: one operating inside the maximal controllable set to the feasible equilibrium set, and the other operating at the N-controllable set to the same equilibrium set. Based on some finite-time convergence results, asymptotic stability of the resulting closed-loop is proved, while recursive feasibility is ensured for any change of the setpoint. The properties and advantages of the proposal have been tested on simulation models.
To promote the widespread use of mobile robots in diverse fields, the performance of trajectory tracking must be ensured. To address the constraints and nonlinear features associated with mobile robot systems, we apply nonlinear model predictive control (MPC) to realize the trajectory tracking of mobile robots. Specifically, to alleviate the online computational complexity of nonlinear MPC, this paper devises a lattice piecewise affine (PWA) approximation method that can approximate both the nonlinear system and control law of explicit nonlinear MPC. The kinematic model of the mobile robot is successively linearized along the trajectory to obtain a linear time-varying description of the system, which is then expressed using a lattice PWA model. Subsequently, the nonlinear MPC problem can be transformed into a series of linear MPC problems. Furthermore, to reduce the complexity of online calculation of multiple linear MPC problems, we approximate the optimal solution of the linear MPC by using the lattice PWA model. That is, for different sampling states, the optimal control inputs are obtained, and lattice PWA approximations are constructed for the state control pairs. Simulations are performed to evaluate the performance of our method in comparison with the linear MPC and explicit linear MPC frameworks. The results show that compared with the explicit linear MPC, our method has a higher online computing speed and can decrease the offline computing time without significantly increasing the tracking error.
The need for control strategies that can address dynamic system uncertainty is becoming increasingly important. In this work, we propose a Model Predictive Control by quantifying the risk of failure in our system model. The proposed control scheme uses a Priced Timed Automata representation of the manufacturing system to promote the fail-safe operation of systems under uncertainties. The proposed method ensures that in case of unforeseen failure(s), the optimization-based control strategy can still achieve the manufacturing system objective. In addition, the proposed strategy establishes a trade-off between minimizing the cost and reducing failure risk to allow the manufacturing system to function effectively in the presence of uncertainties. An example from manufacturing systems is presented to show the application of the proposed control strategy.
Deep learning-based grasp prediction models have become an industry standard for robotic bin-picking systems. To maximize pick success, production environments are often equipped with several end-effector tools that can be swapped on-the-fly, based on the target object. Tool-change, however, takes time. Choosing the order of grasps to perform, and corresponding tool-change actions, can improve system throughput; this is the topic of our work. The main challenge in planning tool change is uncertainty - we typically cannot see objects in the bin that are currently occluded. Inspired by queuing and admission control problems, we model the problem as a Markov Decision Process (MDP), where the goal is to maximize expected throughput, and we pursue an approximate solution based on model predictive control, where at each time step we plan based only on the currently visible objects. Special to our method is the idea of void zones, which are geometrical boundaries in which an unknown object will be present, and therefore cannot be accounted for during planning. Our planning problem can be solved using integer linear programming (ILP). However, we find that an approximate solution based on sparse tree search yields near optimal performance at a fraction of the time. Another question that we explore is how to measure the performance of tool-change planning: we find that throughput alone can fail to capture delicate and smooth behavior, and propose a principled alternative. Finally, we demonstrate our algorithms on both synthetic and real world bin picking tasks.