Grasping is a crucial task in robotics, necessitating tactile feedback and reactive grasping adjustments for robust grasping of objects under various conditions and with differing physical properties. In this paper, we introduce LeTac-MPC, a learning-based model predictive control (MPC) for tactile-reactive grasping. Our approach enables the gripper to grasp objects with different physical properties on dynamic and force-interactive tasks. We utilize a vision-based tactile sensor, GelSight, which is capable of perceiving high-resolution tactile feedback that contains information on the physical properties and states of the grasped object. LeTac-MPC incorporates a differentiable MPC layer designed to model the embeddings extracted by a neural network (NN) from tactile feedback. This design facilitates convergent and robust grasping control at a frequency of 25 Hz. We propose a fully automated data collection pipeline and collect a dataset only using standardized blocks with different physical properties. However, our trained controller can generalize to daily objects with different sizes, shapes, materials, and textures. The experimental results demonstrate the effectiveness and robustness of the proposed approach. We compare LeTac-MPC with two purely model-based tactile-reactive controllers (MPC and PD) and open-loop grasping. Our results show that LeTac-MPC has optimal performance in dynamic and force-interactive tasks and optimal generalizability. We release our code and dataset at https://github.com/ZhengtongXu/LeTac-MPC.
Detention ponds can mitigate flooding and improve water quality by allowing the settlement of pollutants. Typically, they are operated with fully open orifices and weirs (i.e., passive control). Active controls can improve the performance of these systems: orifices can be retrofitted with controlled valves and spillways can have controllable gates. The real-time optimal operation of its hydraulic devices can be achieved with techniques such as Model Predictive Control (MPC). A distributed quasi-2D hydrologic-hydrodynamic coupled with a reservoir flood routing model is developed and integrated with an MPC algorithm to estimate the operation of valves and movable gates. The control optimization problem is adapted to switch from a flood-related algorithm focusing on mitigating floods to a heuristic objective function that aims to increase the detention time when no inflow hydrographs are predicted. The case studies show the potential of applying the methods developed in a catchment in Sao Paulo, Brazil. The performance of MPC compared to alternatives with either fully or partially open valves and gates are tested. Comparisons with HEC-RAS 2D indicate volume and peak flow errors of approximately 1.4% and 0.91% for the watershed module. Simulating two consecutive 10-year storms shows that the MPC strategy can achieve peak flow reductions of 79%. In contrast, passive control has nearly half of the performance. A 1-year continuous simulation results show that the passive scenario with 25% of the valves opened can treat 12% more runoff compared to the developed MPC approach, with an average detention time of approximately 6 hours. For the MPC approach, the average detention time is nearly 14 hours indicating that both control techniques can treat similar volumes; however, the proxy water quality for the MPC approach is enhanced due to the longer detention times.
Model mismatch and process noise are two frequently occurring phenomena that can drastically affect the performance of model predictive control (MPC) in practical applications. We propose a principled way to tune the cost function and the constraints of linear MPC schemes to improve the closed-loop performance and robust constraint satisfaction on uncertain nonlinear dynamics with additive noise. The tuning is performed using a novel MPC tuning algorithm based on backpropagation developed in our earlier work. Using the scenario approach, we provide probabilistic bounds on the likelihood of closed-loop constraint violation over a finite horizon. We showcase the effectiveness of the proposed method on linear and nonlinear simulation examples.
Model Predictive Control (MPC) is a popular control approach due to its ability to consider constraints, including input and state restrictions, while minimizing a cost function. However, in practice, these constraints can result in feasibility issues, either because the system model is not accurate or due to the existence of external disturbances. To mitigate this problem, a solution adopted by the MPC community is the use of soft constraints. In this article, we consider a not-so-typical methodology to encode soft constraints in a particular MPC formulation known as MPC for Tracking (MPCT), which has several advantages when compared to standard MPC formulations. The motivation behind the proposed encoding is to maintain the semi-banded structure of the ingredients of a recently proposed solver for the considered MPCT formulation, thus providing an efficient and fast solver when compared to alternative approaches from the literature. We show numerical results highlighting the benefits of the formulation and the computational efficiency of the solver.
Nonlinear model predictive control (NMPC) is a popular strategy for solving motion planning problems, including obstacle avoidance constraints, in autonomous driving applications. Non-smooth obstacle shapes, such as rectangles, introduce additional local minima in the underlying optimization problem. Smooth over-approximations, e.g., ellipsoidal shapes, limit the performance due to their conservativeness. We propose to vary the smoothness and the related over-approximation by a homotopy. Instead of varying the smoothness in consecutive sequential quadratic programming iterations, we use formulations that decrease the smooth over-approximation from the end towards the beginning of the prediction horizon. Thus, the real-time iterations algorithm is applicable to the proposed NMPC formulation. Different formulations are compared in simulation experiments and shown to successfully improve performance indicators without increasing the computation time.
This paper presents a distributed solution for the problem of collaborative collision avoidance for autonomous inland waterway ships. A two-layer collision avoidance framework that considers inland waterway traffic regulations is proposed to increase navigational safety for autonomous ships. Our approach allows for modifying traffic rules without changing the collision avoidance algorithm, and is based on a novel formulation of model predictive control (MPC) for collision avoidance of ships. This MPC formulation is designed for inland waterway traffic and can handle complex scenarios. The alternating direction method of multipliers is used as a scheme for exchanging and negotiating intentions among ships. Simulation results show that the proposed algorithm can comply with traffic rules. Furthermore, the proposed algorithm can safely deviate from traffic rules when necessary to increase efficiency in complex scenarios.
Neural networks, while powerful, often lack interpretability. Physics-Informed Neural Networks (PINNs) address this limitation by incorporating physics laws into the loss function, making them applicable to solving Ordinary Differential Equations (ODEs) and Partial Differential Equations (PDEs). The recently introduced PINC framework extends PINNs to control applications, allowing for open-ended long-range prediction and control of dynamic systems. In this work, we enhance PINC for modeling highly nonlinear systems such as gas-lifted oil wells. By introducing skip connections in the PINC network and refining certain terms in the ODE, we achieve more accurate gradients during training, resulting in an effective modeling process for the oil well system. Our proposed improved PINC demonstrates superior performance, reducing the validation prediction error by an average of 67% in the oil well application and significantly enhancing gradient flow through the network layers, increasing its magnitude by four orders of magnitude compared to the original PINC. Furthermore, experiments showcase the efficacy of Model Predictive Control (MPC) in regulating the bottom-hole pressure of the oil well using the improved PINC model, even in the presence of noisy measurements.
Optimal decision-making for trajectory tracking in partially observable, stochastic environments where the number of active localization updates -- the process by which the agent obtains its true state information from the sensors -- are limited, presents a significant challenge. Traditional methods often struggle to balance resource conservation, accurate state estimation and precise tracking, resulting in suboptimal performance. This problem is particularly pronounced in environments with large action spaces, where the need for frequent, accurate state data is paramount, yet the capacity for active localization updates is restricted by external limitations. This paper introduces ComTraQ-MPC, a novel framework that combines Deep Q-Networks (DQN) and Model Predictive Control (MPC) to optimize trajectory tracking with constrained active localization updates. The meta-trained DQN ensures adaptive active localization scheduling, while the MPC leverages available state information to improve tracking. The central contribution of this work is their reciprocal interaction: DQN's update decisions inform MPC's control strategy, and MPC's outcomes refine DQN's learning, creating a cohesive, adaptive system. Empirical evaluations in simulated and real-world settings demonstrate that ComTraQ-MPC significantly enhances operational efficiency and accuracy, providing a generalizable and approximately optimal solution for trajectory tracking in complex partially observable environments.
The online implementation of model predictive control for constrained multivariate systems has two main disadvantages: it requires an estimate of the entire model state and an optimisation problem must be solved online. These issues have typically been treated separately. This work proposes an integrated approach for the offline training of an output feedback neural network controller in closed loop. Online this neural network controller computers the plant inputs cheaply using noisy measurements. In addition, the controller can be trained to only make use of certain predefined measurements. Further, a heuristic approach is proposed to perform the automatic selection of important measurements. The proposed method is demonstrated by extensive simulations using a non-linear distillation column model of 50 states.
In recent years, SCADA (Supervisory Control and Data Acquisition) systems have increasingly become the target of cyber attacks. SCADAs are no longer isolated, as web-based applications expose strategic infrastructures to the outside world connection. In a cyber-warfare context, we propose a Model Predictive Control (MPC) architecture with adaptive resilience, capable of guaranteeing control performance in normal operating conditions and driving towards resilience against DoS (controller-actuator) attacks when needed. Since the attackers' goal is typically to maximize the system damage, we assume they solve an adversarial optimal control problem. An adaptive resilience factor is then designed as a function of the intensity function of a Hawkes process, a point process model estimating the occurrence of random events in time, trained on a moving window to estimate the return time of the next attack. We demonstrate the resulting MPC strategy's effectiveness in 2 attack scenarios on a real system with actual data, the regulated Olginate dam of Lake Como.
This paper presents a neural network optimizer with soft-argmax operator to achieve an ecological gearshift strategy in real-time. The strategy is reformulated as the mixed-integer model predictive control (MIMPC) problem to minimize energy consumption. Then the outer convexification is introduced to transform integer variables into relaxed binary controls. To approximate binary solutions properly within training, the soft-argmax operator is applied to the neural network with the fact that all the operations of this scheme are differentiable. Moreover, this operator can help push the relaxed binary variables close to 0 or 1. To evaluate the strategy effect, we deployed it to a 2-speed electric vehicle (EV). In contrast to the mature solver Bonmin, our proposed method not only achieves similar energy-saving effects but also significantly reduces the solution time to meet real-time requirements. This results in a notable energy savings of 6.02% compared to the rule-based method.
To enable autonomous vehicles to perform discretionary lane change amidst the random traffic flow on highways, this paper introduces a decision-making and control method for vehicle lane change based on Model Predictive Control (MPC). This approach divides the driving control of vehicles on highways into two parts: lane-change decision and lane-change control, both of which are solved using the MPC method. In the lanechange decision module, the minimum driving costs for each lane are computed and compared by solving the MPC problem to make lane-change decisions. In the lane-change control module, a dynamic bicycle model is incorporated, and a multi-objective cost function is designed to obtain the optimal control inputs for the lane-change process. Additionally, A long-short term memory (LSTM) model is used to predict the trajectories of surrounding vehicles for both the MPC decision and control modules. The proposed lane-change decision and control method is simulated and validated in a driving simulator under random highway traffic conditions.
The CIRCLES project aims to reduce instabilities in traffic flow, which are naturally occurring phenomena due to human driving behavior. These "phantom jams" or "stop-and-go waves,"are a significant source of wasted energy. Toward this goal, the CIRCLES project designed a control system referred to as the MegaController by the CIRCLES team, that could be deployed in real traffic. Our field experiment leveraged a heterogeneous fleet of 100 longitudinally-controlled vehicles as Lagrangian traffic actuators, each of which ran a controller with the architecture described in this paper. The MegaController is a hierarchical control architecture, which consists of two main layers. The upper layer is called Speed Planner, and is a centralized optimal control algorithm. It assigns speed targets to the vehicles, conveyed through the LTE cellular network. The lower layer is a control layer, running on each vehicle. It performs local actuation by overriding the stock adaptive cruise controller, using the stock on-board sensors. The Speed Planner ingests live data feeds provided by third parties, as well as data from our own control vehicles, and uses both to perform the speed assignment. The architecture of the speed planner allows for modular use of standard control techniques, such as optimal control, model predictive control, kernel methods and others, including Deep RL, model predictive control and explicit controllers. Depending on the vehicle architecture, all onboard sensing data can be accessed by the local controllers, or only some. Control inputs vary across different automakers, with inputs ranging from torque or acceleration requests for some cars, and electronic selection of ACC set points in others. The proposed architecture allows for the combination of all possible settings proposed above. Most configurations were tested throughout the ramp up to the MegaVandertest.
Establishing an execution time certificate in deploying model predictive control (MPC) is a pressing and challenging requirement. As nonlinear MPC (NMPC) results in nonlinear programs, differing from quadratic programs encountered in linear MPC, deriving an execution time certificate for NMPC seems an impossible task. Our prior work \cite{wu2023direct} introduced an input-constrained MPC algorithm with the exact and only \textit{dimension-dependent} (\textit{data-independent}) number of floating-point operations ([flops]). This paper extends it to input-constrained NMPC problems via the real-time iteration (RTI) scheme, which results in \textit{data-varying} (but \textit{dimension-invariant}) input-constrained MPC problems. Therefore, applying our previous algorithm can certify the execution time based on the assumption that processors perform fixed [flops] in constant time. As the RTI-based scheme generally results in MPC with a long prediction horizon, this paper employs the efficient factorized Riccati recursion, whose computational cost scales linearly with the prediction horizon, to solve the Newton system at each iteration. The execution-time certified capability of the algorithm is theoretically and numerically validated through a case study involving nonlinear control of the chaotic Lorenz system.
Feedback control synthesis for large-scale particle systems is reviewed in the framework of model predictive control (MPC). The high-dimensional character of collective dynamics hampers the performance of traditional MPC algorithms based on fast online dynamic optimization at every time step. Two alternatives to MPC are proposed. First, the use of supervised learning techniques for the offline approximation of optimal feedback laws is discussed. Then, a procedure based on sequential linearization of the dynamics based on macroscopic quantities of the particle ensemble is reviewed. Both approaches circumvent the online solution of optimal control problems enabling fast, real-time, feedback synthesis for large-scale particle systems. Numerical experiments assess the performance of the proposed algorithms.
This study presents a novel method for microstructure control in closed die hot forging that combines Model Predictive Control (MPC) with a developed machine learning model called DeepForge. DeepForge uses an architecture that combines 1D convolutional neural networks and gated recurrent units. It uses surface temperature measurements of a workpiece as input to predict microstructure changes during forging. The paper also details DeepForge's architecture and the finite element simulation model used to generate the data set, using a three-stroke forging process. The results demonstrate DeepForge's ability to predict microstructure with a mean absolute error of 0.4$\pm$0.3%. In addition, the study explores the use of MPC to adjust inter-stroke wait times, effectively counteracting temperature disturbances to achieve a target grain size of less than 35 microns within a specific 2D region of the workpiece. These results are then verified experimentally, demonstrating a significant step towards improved control and quality in forging processes where temperature can be used as an additional degree of freedom in the process.
In coastal river systems, frequent floods, often occurring during major storms or king tides, pose a severe threat to lives and property. However, these floods can be mitigated or even prevented by strategically releasing water before extreme weather events with hydraulic structures such as dams, gates, pumps, and reservoirs. A standard approach used by local water management agencies is the "rule-based" method, which specifies predetermined pre-releases of water based on historical and time-tested human experience, but which tends to result in excess or inadequate water release. The model predictive control (MPC), a physics-based model for prediction, is an alternative approach, albeit involving computationally intensive calculations. In this paper, we propose a Forecast Informed Deep Learning Architecture, FIDLAR, to achieve rapid and optimal flood management with precise water pre-releases. FIDLAR seamlessly integrates two neural network modules: one called the Flood Manager, which is responsible for generating water pre-release schedules, and another called the Flood Evaluator, which assesses these generated schedules. The Evaluator module is pre-trained separately, and its gradient-based feedback is used to train the Manager model, ensuring optimal water pre-releases. We have conducted experiments using FIDLAR with data from a flood-prone coastal area in South Florida, particularly susceptible to frequent storms. Results show that FIDLAR is several orders of magnitude faster than currently used physics-based approaches while outperforming baseline methods with improved water pre-release schedules.
Robust Model Predictive Control (MPC) for nonlinear systems is a problem that poses significant challenges as highlighted by the diversity of approaches proposed in the last decades. Often compromises with respect to computational load, conservatism, generality, or implementation complexity have to be made, and finding an approach that provides the right balance is still a challenge to the research community. This work provides a contribution by proposing a novel shrinking-horizon robust MPC formulation for nonlinear discrete-time systems. By explicitly accounting for how disturbances and linearization errors are propagated through the nonlinear dynamics, a constraint tightening-based formulation is obtained, with guarantees of robust constraint satisfaction. The proposed controller relies on iteratively solving a Nonlinear Program (NLP) to simultaneously optimize system operation and the required constraint tightening. Numerical experiments show the effectiveness of the proposed controller with three different choices of NLP solvers as well as significantly improved computational speed, better scalability, and generally reduced conservatism when compared to an existing technique from the literature.
The optimal operation of transportation systems is often susceptible to unexpected disruptions, such as traffic accidents and social events. Many established control strategies relying on mathematical models can struggle with real-world disruptions, leading to a significant deviation from their anticipated efficiency. This work applies the cutting-edge concept of antifragility to design a traffic control strategy for urban road networks against disruptions. Antifragility sets itself apart from robustness, resilience, and reliability as it represents a system's ability to not only withstand stressors, shocks, and volatility but also to thrive and enhance performance in the presence of such adversarial events. Incorporating antifragile terms composed of traffic state derivatives and redundancy, a model-free deep reinforcement learning algorithm is developed and subsequently evaluated in a two-region cordon-shaped urban traffic perimeter network. Promising results highlight (a) the superior performance of the proposed algorithm compared to the state-of-the-art methods under incremental magnitude of disruptions, (b) distribution skewness as the antifragility indicator demonstrating its relative antifragility, and (c) its effectiveness under limited observability due to real-world data availability constraints. The proposed antifragile methodology is generalizable and holds potential for application beyond perimeter control, offering integration into systems exposed to disruptions across various disciplines.
Heat pump and thermal energy storage (HPTES) systems, which are widely utilized in modern buildings for providing domestic hot water, contribute to a large share of household electricity consumption. With the increasing integration of renewable energy sources (RES) into modern power grids, demand-side management (DSM) becomes crucial for balancing power generation and consumption by adjusting end users' power consumption. This paper explores an energy flexible Model Predictive Control (MPC) design for a class of HPTES systems to facilitate demand-side management. The proposed DSM strategy comprises two key components: i) flexibility assessment, and ii) flexibility exploitation. Firstly, for flexibility assessment, a tailored MPC formulation, supplemented by a set of auxiliary linear constraints, is developed to quantitatively assess the flexibility potential inherent in HPTES systems. Subsequently, in flexibility exploitation, the energy flexibility is effectively harnessed in response to feasible demand response (DR) requests, which can be formulated as a standard mixed-integer MPC problem. Numerical experiments, based on a real-world HPTES installation, are conducted to demonstrate the efficacy of the proposed design.