The multi-timestep command governor (MCG) is an add-on algorithm that enforces constraints by modifying, at each timestep, the reference command to a pre-stabilized control system. The MCG can be interpreted as a Model-Predictive Control scheme operating on the reference command. The implementation of MCG on nonlinear systems carries a heavy computational burden as it requires solving a nonlinear program with multiple decision variables at each timestep. This paper proposes a less computationally demanding alternative, based on approximating the MCG control law using a neural network (NN) trained on offline data. However, since the NN output may not always be constraint-admissible due to training errors, its output is adjusted using a sensitivity-based method. We thus refer to the resulting control strategy as the neural network-based MCG (NN-MCG). As validation, the proposed controller is applied as a load governor for constraint management in an automotive fuel cell system. It is shown that the proposed strategy is significantly more computationally efficient than the traditional MCG, while achieving nearly identical performance if the NN is well-trained.
This work proposes an adaptive output feedback model predictive control (MPC) framework for uncertain systems subject to external disturbances. In the absence of exact knowledge about the plant parameters and complete state measurements, the MPC optimization problem is reformulated in terms of their estimates derived from a suitably designed robust adaptive observer. The MPC routine returns a homothetic tube for the state estimate trajectory. Sets that characterize the state estimation errors are then added to the homothetic tube sections, resulting in a larger tube containing the true state trajectory. The two-tier tube architecture provides robustness to uncertainties due to imperfect parameter knowledge, external disturbances, and incomplete state information. Additionally, recursive feasibility and robust exponential stability are guaranteed and validated using a numerical example.
This paper presents a novel contingency planning framework that integrates learning-based multi-modal predictions of traffic participants into Branch Model Predictive Control (MPC). Leveraging reachability analysis, we address the computational challenges associated with Branch MPC by organizing the multitude of predictions into driving corridors. Analyzing the overlap between these corridors, their number can be reduced through pruning and clustering while ensuring safety since all prediction modes are preserved. These processed corridors directly correspond to the distinct branches of the scenario tree and provide an efficient constraint representation for the Branch MPC. We further utilize the reachability for determining maximum feasible decision postponing times, ensuring that branching decisions remain executable. Qualitative and quantitative evaluations demonstrate significantly reduced computational complexity and enhanced safety and comfort.
Model Predictive Control (MPC) is a widely known control method that has proved to be particularly effective in multivariable and constrained control. Closed-loop stability and recursive feasibility can be guaranteed by employing accurate models in prediction and suitable terminal ingredients, i.e. the terminal cost function and the terminal constraint. Issues might arise in case of model mismatches or perturbed systems, as the state predictions could be inaccurate, and nonlinear systems for which the computation of the terminal ingredients can result challenging. In this manuscript, we exploit the properties of component-wise uniformly continuous and stabilizable systems to introduce a robust contraction-based MPC for the regulation of nonlinear perturbed systems, that employs an easy-to-design terminal cost function, does not make use of terminal constraints, and selects the shortest prediction horizon that guarantees the stability of the closed-loop system.
Due to its state-of-the-art estimation performance complemented by rigorous and non-conservative uncertainty bounds, Gaussian process regression is a popular tool for enhancing dynamical system models and coping with their inaccuracies. This has enabled a plethora of successful implementations of Gaussian process-based model predictive control in a variety of applications over the last years. However, despite its evident practical effectiveness, there are still many open questions when attempting to analyze the associated optimal control problem theoretically and to exploit the full potential of Gaussian process regression in view of safe learning-based control. The contribution of this review is twofold. The first is to survey the available literature on the topic, highlighting the major theoretical challenges such as (i) addressing scalability issues of Gaussian process regression; (ii) taking into account the necessary approximations to obtain a tractable MPC formulation; (iii) including online model updates to refine the dynamics description, exploiting data collected during operation. The second is to provide an extensive discussion of future research directions, collecting results on uncertainty quantification that are related to (but yet unexploited in) optimal control, among others. Ultimately, this paper provides a toolkit to study and advance Gaussian process-based model predictive control.
The fields of MPC and RL consider two successful control techniques for Markov decision processes. Both approaches are derived from similar fundamental principles, and both are widely used in practical applications, including robotics, process control, energy systems, and autonomous driving. Despite their similarities, MPC and RL follow distinct paradigms that emerged from diverse communities and different requirements. Various technical discrepancies, particularly the role of an environment model as part of the algorithm, lead to methodologies with nearly complementary advantages. Due to their orthogonal benefits, research interest in combination methods has recently increased significantly, leading to a large and growing set of complex ideas leveraging MPC and RL. This work illuminates the differences, similarities, and fundamentals that allow for different combination algorithms and categorizes existing work accordingly. Particularly, we focus on the versatile actor-critic RL approach as a basis for our categorization and examine how the online optimization approach of MPC can be used to improve the overall closed-loop performance of a policy.
Drone racing involves high-speed navigation of three-dimensional paths, posing a substantial challenge in control engineering. This study presents a game-theoretic control framework, the nonlinear receding-horizon differential game (NRHDG), designed for competitive drone racing. NRHDG enhances robustness in adversarial settings by predicting and countering an opponent's worst-case behavior in real time. It extends standard nonlinear model predictive control (NMPC), which otherwise assumes a fixed opponent model. First, we develop a novel path-following formulation based on projection point dynamics, eliminating the need for costly distance minimization. Second, we propose a potential function that allows each drone to switch between overtaking and obstructing maneuvers based on real-time race situations. Third, we establish a new performance metric to evaluate NRHDG with NMPC under race scenarios. Simulation results demonstrate that NRHDG outperforms NMPC in terms of both overtaking efficiency and obstructing capabilities.
Model predictive control (MPC) is a popular control engineering practice, but requires a sound knowledge of the model. Model-free predictive control (MFPC), a burning issue today, also related to reinforcement learning (RL) in AI, is reformulated here via a linear differential equation with constant coefficients, thanks to a new perspective on optimal control combined with recent advances in the field of model-free control (MFC). It is replacing Dynamic Programming, the Hamilton-Jacobi-Bellman equation, and Pontryagin's Maximum Principle. The computing burden is low. The implementation is straightforward. Two nonlinear examples, a chemical reactor and a two tank system, are illustrating our approach. A comparison with the HEOL setting, where some expertise of the process model is needed, shows only a slight superiority of the later. A recent identification of the two tank system via a complex ANN architecture might indicate that a full modeling and the corresponding machine learning mechanism are not always necessary neither in control, nor, more generally, in AI.
The use of connected automated vehicle (CAV) is advocated to mitigate traffic oscillations in mixed traffic flow consisting of CAVs and human driven vehicles (HDVs). This study proposes an adaptive deep Koopman predictive control framework (AdapKoopPC) for regulating mixed traffic flow. Firstly, a Koopman theory-based adaptive trajectory prediction deep network (AdapKoopnet) is designed for modeling HDVs car-following behavior. AdapKoopnet enables the representation of HDVs behavior by a linear model in a high-dimensional space. Secondly, the model predictive control is employed to smooth the mixed traffic flow, where the combination of the linear dynamic model of CAVs and linear prediction blocks from AdapKoopnet is embedded as the predictive model into the AdapKoopPC. Finally, the predictive performance of the prosed AdapKoopnet is verified using the HighD naturalistic driving dataset. Furthermore, the control performance of AdapKoopPC is validated by the numerical simulations. Results demonstrate that the AdapKoopnet provides more accuracy HDVs predicted trajectories than the baseline nonlinear models. Moreover, the proposed AdapKoopPC exhibits more effective control performance with less computation cost compared with baselines in mitigating traffic oscillations, especially at the low CAVs penetration rates. The code of proposed AdapKoopPC is open source.
Soft robots can revolutionize several applications with high demands on dexterity and safety. When operating these systems, real-time estimation and control require fast and accurate models. However, prediction with first-principles (FP) models is slow, and learned black-box models have poor generalizability. Physics-informed machine learning offers excellent advantages here, but it is currently limited to simple, often simulated systems without considering changes after training. We propose physics-informed neural networks (PINNs) for articulated soft robots (ASRs) with a focus on data efficiency. The amount of expensive real-world training data is reduced to a minimum -- one dataset in one system domain. Two hours of data in different domains are used for a comparison against two gold-standard approaches: In contrast to a recurrent neural network, the PINN provides a high generalizability. The prediction speed of an accurate FP model is exceeded with the PINN by up to a factor of 467 at slightly reduced accuracy. This enables nonlinear model predictive control (MPC) of a pneumatic ASR. Accurate position tracking with the MPC running at 47 Hz is achieved in six dynamic experiments.
Controlling spacecraft near asteroids in deep space comes with many challenges. The delays involved necessitate heavy usage of limited onboard computation resources while fuel efficiency remains a priority to support the long loiter times needed for gathering data. Additionally, the difficulty of state determination due to the lack of traditional reference systems requires a guidance, navigation, and control (GNC) pipeline that ideally is both computationally and fuel-efficient, and that incorporates a robust state determination system. In this paper, we propose an end-to-end algorithm utilizing neural networks to generate near-optimal control commands from raw sensor data, as well as a hybrid model predictive control (MPC) guided imitation learning controller delivering improvements in computational efficiency over a traditional MPC controller.
Intelligent transportation systems have recently emerged to address the growing interest for safer, more efficient, and sustainable transportation solutions. In this direction, this paper presents distributed algorithms for control and optimization over vehicular networks. First, we formulate the autonomous vehicle platooning framework based on model-predictive-control (MPC) strategies and present its objective optimization as a cooperative quadratic cost function. Then, we propose a distributed algorithm to locally optimize this objective at every vehicle subject to data quantization over the communication network of vehicles. In contrast to most existing literature that assumes ideal communication channels, log-scale data quantization over the network is addressed in this work, which is more realistic and practical. In particular, we show by simulation that the proposed log-quantized algorithm reaches optimal convergence with less residual and optimality gap. This outperforms the existing literature considering uniform quantization which leads to a large optimality gap and residual.
In this paper, we propose an economic nonlinear model predictive control (MPC) algorithm for district heating networks (DHNs). The proposed method features prosumers, multiple producers, and storage systems, which are essential components of 4th generation DHNs. These networks are characterized by their ability to optimize their operations, aiming to reduce supply temperatures, accommodate distributed heat sources, and leverage the flexibility provided by thermal inertia and storage, all crucial for achieving a fossil-fuel-free energy supply. Developing a smart energy management system to accomplish these goals requires detailed models of highly complex nonlinear systems and computational algorithms able to handle large-scale optimization problems. To address this, we introduce a graph-based optimization-oriented model that efficiently integrates distributed producers, prosumers, storage buffers, and bidirectional pipe flows, such that it can be implemented in a real-time MPC setting. Furthermore, we conduct several numerical experiments to evaluate the performance of the proposed algorithms in closed-loop. Our findings demonstrate that the MPC methods achieved up to 9% cost improvement over traditional rule-based controllers while better maintaining system constraints.
The evolution of information and communication technologies has yielded the means of sharing measurements and other information in an efficient and flexible way, which has enabled the size and complexity of control applications to increase. At the same time, the improvements in the computational and communicational capabilities of control devices have fostered the development of noncentralized control architectures, already motivated by the inherent structural constraints of large-scale systems. Computer-based control approaches such as model predictive control (MPC) are visible beneficiaries of these advances and have registered a significant growth regarding both theoretical and applied fields. Coalitional control focuses on the local interests that motivate the controllers to assemble, an aspect so far rarely contemplated in the distributed control literature. This article presents the main concepts and challenges in coalitional control, and the links with cooperative network game theory.
We present a hierarchical control scheme for large-scale systems whose components can exchange information through a data network. The main goal of the supervisory layer is to find the best compromise between control performance and communicational costs by actively modifying the network topology. The actions taken at the supervisory layer alter the control agents' knowledge of the complete system, and the set of agents with which they can communicate. Each group of linked subsystems, or coalition, is independently controlled based on a decentralized model predictive control (MPC) scheme, managed at the bottom layer. Hard constraints on the inputs are imposed, while soft constraints on the states are considered to avoid feasibility issues. The performance of the proposed control scheme is validated on a model of the Dez irrigation canal, implemented on the accurate simulator for water systems SOBEK. Finally, the results are compared with those obtained using a centralized MPC controller.
This paper presents an optic flow-guided approach for achieving soft landings by resource-constrained unmanned aerial vehicles (UAVs) on dynamic platforms. An offline data-driven linear model based on Koopman operator theory is developed to describe the underlying (nonlinear) dynamics of optic flow output obtained from a single monocular camera that maps to vehicle acceleration as the control input. Moreover, a novel adaptation scheme within the Koopman framework is introduced online to handle uncertainties such as unknown platform motion and ground effect, which exert a significant influence during the terminal stage of the descent process. Further, to minimize computational overhead, an event-based adaptation trigger is incorporated into an event-driven Model Predictive Control (MPC) strategy to regulate optic flow and track a desired reference. A detailed convergence analysis ensures global convergence of the tracking error to a uniform ultimate bound while ensuring Zeno-free behavior. Simulation results demonstrate the algorithm's robustness and effectiveness in landing on dynamic platforms under ground effect and sensor noise, which compares favorably to non-adaptive event-triggered and time-triggered adaptive schemes.
Recent works in Learning-Based Model Predictive Control of dynamical systems show impressive sample complexity performances using criteria from Information Theory to accelerate the learning procedure. However, the sequential exploration opportunities are limited by the system local state, restraining the amount of information of the observations from the current exploration trajectory. This article resolves this limitation by introducing temporal abstraction through the framework of Semi-Markov Decision Processes. The framework increases the total information of the gathered data for a fixed sampling budget, thus reducing the sample complexity.
This paper focuses on the problem of collaborative collision avoidance for autonomous inland ships. Two solutions are provided to solve the problem in a distributed manner. We first present a distributed model predictive control (MPC) algorithm that allows ships to directly negotiate their intention to avoid collision in a synchronous communication framework. Moreover, we introduce a new approach to shape the ship's behavior to follow the waterway traffic regulations. The conditional convergence toward a stationary solution of this algorithm is guaranteed by the theory of the Alternating Direction Method of Multipliers (ADMM). To overcome the problem of asynchronous communication between ships, we adopt a new asynchronous nonlinear ADMM and present an asynchronous distributed MPC algorithm based on it. Several simulations and field experiments show that the proposed algorithms can prevent ship collisions even in complex scenarios.
In this paper, we present an early software integrating Reinforcement Learning (RL) with Model Predictive Control (MPC). Our aim is to make recent theoretical contributions from the literature more accessible to both the RL and MPC communities. We combine standard software tools developed by the RL community, such as Gymnasium, stable-baselines3, or CleanRL with the acados toolbox, a widely-used software package for efficient MPC algorithms. Our core contribution is MPC4RL, an open-source Python package that supports learning-enhanced MPC schemes for existing acados implementations. The package is designed to be modular, extensible, and user-friendly, facilitating the tuning of MPC algorithms for a broad range of control problems. It is available on GitHub.
In many mechatronic applications, controller input costs are negligible and time optimality is of great importance to maximize the productivity by executing fast positioning maneuvers. As a result, the obtained control input has mostly a bang-bang nature, which excite undesired mechanical vibrations, especially in systems with flexible structures. This paper tackles the time-optimal control problem and proposes a novel approach, which explicitly addresses the vibrational behavior in the context of the receding horizon technique. Such technique is a key feature, especially for systems with a time-varying vibrational behavior. In the context of model predictive control (MPC), vibrational behavior is predicted and coped in a soft-constrained formulation, which penalize any violation of undesired vibrations. This formulation enlarges the feasibility on a wide operating range in comparison with a hard-constrained formulation. The closed-loop performance of this approach is demonstrated on a numerical example of stacker crane with high degree of flexibility.