Sampling-based Model Predictive Control (MPC) has been a practical and effective approach in many domains, notably model-based reinforcement learning, thanks to its flexibility and parallelizability. Despite its appealing empirical performance, the theoretical understanding, particularly in terms of convergence analysis and hyperparameter tuning, remains absent. In this paper, we characterize the convergence property of a widely used sampling-based MPC method, Model Predictive Path Integral Control (MPPI). We show that MPPI enjoys at least linear convergence rates when the optimization is quadratic, which covers time-varying LQR systems. We then extend to more general nonlinear systems. Our theoretical analysis directly leads to a novel sampling-based MPC algorithm, CoVariance-Optimal MPC (CoVo-MPC) that optimally schedules the sampling covariance to optimize the convergence rate. Empirically, CoVo-MPC significantly outperforms standard MPPI by 43-54% in both simulations and real-world quadrotor agile control tasks. Videos and Appendices are available at \url{https://lecar-lab.github.io/CoVO-MPC/}.
Computational efficiency and robustness are essential in process modeling, optimization, and control for real-world engineering applications. While neural network-based approaches have gained significant attention in recent years, conventional neural networks often fail to address these two critical aspects simultaneously or even independently. Inspired by natural physical systems and established literature, input convex architectures are known to enhance computational efficiency in optimization tasks, whereas Lipschitz-constrained architectures improve robustness. However, combining these properties within a single model requires careful review, as inappropriate methods for enforcing one property can undermine the other. To overcome this, we introduce a novel network architecture, termed Input Convex Lipschitz Recurrent Neural Networks (ICLRNNs). This architecture seamlessly integrates the benefits of convexity and Lipschitz continuity, enabling fast and robust neural network-based modeling and optimization. The ICLRNN outperforms existing recurrent units in both computational efficiency and robustness. Additionally, it has been successfully applied to practical engineering scenarios, such as modeling and control of chemical process and the modeling and real-world solar irradiance prediction for solar PV system planning at LHT Holdings in Singapore. Source code is available at https://github.com/killingbear999/ICLRNN.
This paper proposes a comprehensive hierarchical control framework for autonomous decision-making arising in robotics and autonomous systems. In a typical hierarchical control architecture, high-level decision making is often characterised by discrete state and decision/control sets. However, a rational decision is usually affected by not only the discrete states of the autonomous system, but also the underlying continuous dynamics even the evolution of its operational environment. This paper proposes a holistic and comprehensive design process and framework for this type of challenging problems, from new modelling and design problem formulation to control design and stability analysis. It addresses the intricate interplay between traditional continuous systems dynamics utilized at the low levels for control design and discrete Markov decision processes (MDP) for facilitating high-level decision making. We model the decision making system in complex environments as a hybrid system consisting of a controlled MDP and autonomous (i.e. uncontrolled) continuous dynamics. Consequently, the new formulation is called as hybrid Markov decision process (HMDP). The design problem is formulated with a focus on ensuring both safety and optimality while taking into account the influence of both the discrete and continuous state variables of different levels. With the help of the model predictive control (MPC) concept, a decision maker design scheme is proposed for the proposed hybrid decision making model. By carefully designing key ingredients involved in this scheme, it is shown that the recursive feasibility and stability of the proposed autonomous decision making scheme are guaranteed. The proposed framework is applied to develop an autonomous lane changing system for intelligent vehicles.
In this paper, a computationally lightweight algorithm is introduced for hybrid PV/Battery/Load systems that is price responsive, responds fast, does not require powerful hardware, and considers the operational limitations of the system. The method is applied to two buildings equipped with PV and battery. Simulation results show that the method can give results that are up to 3.9% more expensive than the Model predictive control (MPC) approach while the runtime of the program is up to 1000 times less than the MPC. Also, while the runtime of the proposed method is in the range of the self-consumption maximization (SCM) approach as the fastest method, its electricity cost is about 3.2% cheaper than the SCM method. Simulation results also show that in case of providing grid services by the battery the difference between electricity cost of the proposed approach and MPC can reduce which makes the method good for such applications.
Search for the optimizer in computationally demanding model predictive control (MPC) setups can be facilitated by Cloud as a service provider in cyber-physical systems. This advantage introduces the risk that Cloud can obtain unauthorized access to the privacy-sensitive parameters of the system and cost function. To solve this issue, i.e., preventing Cloud from accessing the parameters while benefiting from Cloud computation, random affine transformations provide an exact yet light weight in computation solution. This research deals with analyzing privacy preserving properties of these transformations when they are adopted for MPC problems. We consider two common strategies for outsourcing the optimization required in MPC problems, namely separate and dense forms, and establish that random affine transformations utilized in these forms are vulnerable to side-knowledge from Cloud. Specifically, we prove that the privacy guarantees of these methods and their extensions for separate form are undermined when a mild side-knowledge about the problem in terms of structure of MPC cost function is available. In addition, while we prove that outsourcing the MPC problem in the dense form inherently leads to some degree of privacy for the system and cost function parameters, we also establish that affine transformations applied to this form are nevertheless prone to be undermined by a Cloud with mild side-knowledge. Numerical simulations confirm our results.
Model Predictive Control (MPC) is a versatile approach capable of accommodating diverse control requirements that holds significant promise for a broad spectrum of industrial applications. Noteworthy challenges associated with MPC include the substantial computational burden, which is sometimes considered excessive even for linear systems. Recently, a rapid computation method that guides the input toward convergence with the optimal control problem solution by employing primal-dual gradient (PDG) dynamics as a controller has been proposed for linear MPCs. However, stability has been ensured under the assumption that the controller is a continuous-time system, leading to potential instability when the controller undergoes discretization and is implemented as a sampled-data system. In this paper, we propose a discrete-time dynamical controller, incorporating specific modifications to the PDG approach, and present stability conditions relevant to the resulting sampled-data system. Additionally, we introduce an extension designed to enhance control performance, that was traded off in the original. Numerical examples substantiate that our proposed method, which can be executed in only 1 $\mu$s in a standard laptop, not only ensures stability with considering sampled-data implementation but also effectively enhances control performance.
This paper presents a new approach to predict the occupancy for building energy systems (BES). A Gaussian Process (GP) is used to model the occupancy and is represented as a state space model that is equivalent to the full GP if Kalman filtering and smoothing is used. The combination of GPs and mechanistic models is called Latent Force Model (LFM). An LFM-based model predictive control (MPC) concept for BES is presented that benefits from the extrapolation capability of mechanistic models and the learning ability of GPs to predict the occupancy within the building. Simulations with EnergyPlus and a comparison with real-world data from the Bosch Research Campus in Renningen show that a reduced energy demand and thermal discomfort can be obtained with the LFM-based MPC scheme by accounting for the predicted stochastic occupancy.
The extensive use of distributed vehicle platoon controllers has resulted in several benefits for transportation systems, such as increased traffic flow, fuel efficiency, and decreased pollution. The rising reliance on interconnected systems and communication networks, on the other hand, exposes these controllers to potential cyber-attacks, which may compromise their safety and functionality. This thesis aims to improve the security of distributed vehicle platoon controllers by investigating attack scenarios and assessing their influence on system performance. Various attack techniques, including man-in-the-middle (MITM) and false data injection (FDI), are simulated using Model Predictive Control (MPC) controller to identify vulnerabilities and weaknesses of the platoon controller. Countermeasures are offered and tested, that includes attack analysis and reinforced communication protocols using Machine Learning techniques for detection. The findings emphasize the significance of integrating security issues into their design and implementation, which helps to construct safe and resilient distributed platoon controllers.
This article investigates the problem of controlling linear time-invariant systems subject to time-varying and a priori unknown cost functions, state and input constraints, and exogenous disturbances. We combine the online convex optimization framework with tools from robust model predictive control to propose an algorithm that is able to guarantee robust constraint satisfaction. The performance of the closed loop emerging from application of our framework is studied in terms of its dynamic regret, which is proven to be bounded linearly by the variation of the cost functions and the magnitude of the disturbances. We corroborate our theoretical findings and illustrate implementational aspects of the proposed algorithm by a numerical case study on a tracking control problem of an autonomous vehicle.
We use Koopman theory for data-driven model reduction of nonlinear dynamical systems with controls. We propose generic model structures combining delay-coordinate encoding of measurements and full-state decoding to integrate reduced Koopman modeling and state estimation. We present a deep-learning approach to train the proposed models. A case study demonstrates that our approach provides accurate control models and enables real-time capable nonlinear model predictive control of a high-purity cryogenic distillation column.
We present a chance-constrained model predictive control (MPC) framework under Gaussian mixture model (GMM) uncertainty. Specifically, we consider the uncertainty that arises from predicting future behaviors of moving obstacles, which may exhibit multiple modes (for example, turning left or right). To address the multi-modal uncertainty distribution, we propose three MPC formulations: nominal chance-constrained planning, robust chance-constrained planning, and contingency planning. We prove that closed-loop trajectories generated by the three planners are safe. The approaches differ in conservativeness and performance guarantee. In particular, the robust chance-constrained planner is recursively feasible under certain assumptions on the propagation of prediction uncertainty. On the other hand, the contingency planner generates a less conservative closed-loop trajectory than the nominal planner. We validate our planners using state-of-the-art trajectory prediction algorithms in autonomous driving simulators.
Sampling-based model predictive control (MPC) has found significant success in optimal control problems with non-smooth system dynamics and cost function. Many machine learning-based works proposed to improve MPC by a) learning or fine-tuning the dynamics/ cost function, or b) learning to optimize for the update of the MPC controllers. For the latter, imitation learning-based optimizers are trained to update the MPC controller by mimicking the expert demonstrations, which, however, are expensive or even unavailable. More significantly, many sequential decision-making problems are in non-stationary environments, requiring that an optimizer should be adaptable and generalizable to update the MPC controller for solving different tasks. To address those issues, we propose to learn an optimizer based on meta-reinforcement learning (RL) to update the controllers. This optimizer does not need expert demonstration and can enable fast adaptation (e.g., few-shots) when it is deployed in unseen control tasks. Experimental results validate the effectiveness of the learned optimizer regarding fast adaptation.
The enduring challenge in the field of artificial intelligence has been the control of systems to achieve desired behaviours. While for systems governed by straightforward dynamics equations, methods like Linear Quadratic Regulation (LQR) have historically proven highly effective, most real-world tasks, which require a general problem-solver, demand world models with dynamics that cannot be easily described by simple equations. Consequently, these models must be learned from data using neural networks. Most model predictive control (MPC) algorithms designed for visual world models have traditionally explored gradient-free population-based optimisation methods, such as Cross Entropy and Model Predictive Path Integral (MPPI) for planning. However, we present an exploration of a gradient-based alternative that fully leverages the differentiability of the world model. In our study, we conduct a comparative analysis between our method and other MPC-based alternatives, as well as policy-based algorithms. In a sample-efficient setting, our method achieves on par or superior performance compared to the alternative approaches in most tasks. Additionally, we introduce a hybrid model that combines policy networks and gradient-based MPC, which outperforms pure policy based methods thereby holding promise for Gradient-based planning with world models in complex real-world tasks.
We propose a data-driven receding-horizon control method dealing with the chance-constrained output-tracking problem of unknown stochastic linear time-invariant (LTI) systems with partial state observation. The proposed method takes into account the statistics of the process noise, the measurement noise and the uncertain initial condition, following an analogous framework to Stochastic Model Predictive Control (SMPC), but does not rely on the use of a parametric system model. As such, our receding-horizon algorithm produces a sequence of closed-loop control policies for predicted time steps, as opposed to a sequence of open-loop control actions. Under certain conditions, we establish that our proposed data-driven control method produces identical control inputs as that produced by the associated model-based SMPC. Simulation results on a grid-connected power converter are provided to illustrate the performance benefits of our methodology.
Model Predictive Control (MPC) is a powerful method for complex system regulation, but its reliance on an accurate model poses many limitations in real-world applications. Data-driven predictive control (DDPC) aims at overcoming this limitation, by relying on historical data to provide information on the plant to be controlled. In this work, we present a unified stochastic framework for direct DDPC, where control actions are obtained by optimizing the Final Control Error (FCE), which is directly computed from available data only and automatically weighs the impact of uncertainty on the control objective. Our framework allows us to establish a separation principle for Predictive Control, elucidating the role that predictive models and their uncertainty play in DDPC. Moreover, it generalizes existing DDPC methods, like regularized Data-enabled Predictive Control (DeePC) and $\gamma$-DDPC, providing a path toward noise-tolerant data-based control with rigorous optimality guarantees. The theoretical investigation is complemented by a series of experiments (code available on GitHub: https://github.com/marcofabris92/a-separation-principle-in-d3pc), revealing that the proposed method consistently outperforms or, at worst, matches existing techniques without requiring tuning regularization parameters as other methods do.
Objective. Precise control of neural systems is essential to experimental investigations of how the brain controls behavior and holds the potential for therapeutic manipulations to correct aberrant network states. Model predictive control, which employs a dynamical model of the system to find optimal control inputs, has promise for dealing with the nonlinear dynamics, high levels of exogenous noise, and limited information about unmeasured states and parameters that are common in a wide range of neural systems. However, the challenge still remains of selecting the right model, constraining its parameters, and synchronizing to the neural system. Approach. As a proof of principle, we used recent advances in data-driven forecasting to construct a nonlinear machine-learning model of a Hodgkin-Huxley type neuron when only the membrane voltage is observable and there are an unknown number of intrinsic currents. Main Results. We show that this approach is able to learn the dynamics of different neuron types and can be used with MPC to force the neuron to engage in arbitrary, researcher-defined spiking behaviors. Significance. To the best of our knowledge, this is the first application of nonlinear MPC of a conductance-based model where there is only realistically limited information about unobservable states and parameters.
We propose an adaptive model-predictive controller that balances driving the system to a goal state and seeking system observations that are informative with respect to the parameters of a nonlinear autoregressive exogenous model. The controller's objective function is derived from an expected free energy functional and contains information-theoretic terms expressing uncertainty over model parameters and output predictions. Experiments illustrate how parameter uncertainty affects the control objective and evaluate the proposed controller for a pendulum swing-up task.
Autonomous Vehicles (AVs) have attracted significant attention in recent years and Reinforcement Learning (RL) has shown remarkable performance in improving the autonomy of vehicles. In that regard, the widely adopted Model-Free RL (MFRL) promises to solve decision-making tasks in connected AVs (CAVs), contingent on the readiness of a significant amount of data samples for training. Nevertheless, it might be infeasible in practice and possibly lead to learning instability. In contrast, Model-Based RL (MBRL) manifests itself in sample-efficient learning, but the asymptotic performance of MBRL might lag behind the state-of-the-art MFRL algorithms. Furthermore, most studies for CAVs are limited to the decision-making of a single AV only, thus underscoring the performance due to the absence of communications. In this study, we try to address the decision-making problem of multiple CAVs with limited communications and propose a decentralized Multi-Agent Probabilistic Ensembles with Trajectory Sampling algorithm MA-PETS. In particular, in order to better capture the uncertainty of the unknown environment, MA-PETS leverages Probabilistic Ensemble (PE) neural networks to learn from communicated samples among neighboring CAVs. Afterwards, MA-PETS capably develops Trajectory Sampling (TS)-based model-predictive control for decision-making. On this basis, we derive the multi-agent group regret bound affected by the number of agents within the communication range and mathematically validate that incorporating effective information exchange among agents into the multi-agent learning scheme contributes to reducing the group regret bound in the worst case. Finally, we empirically demonstrate the superiority of MA-PETS in terms of the sample efficiency comparable to MFBL.
Safety guarantees are vital in many control applications, such as robotics. Model predictive control (MPC) provides a constructive framework for controlling safety-critical systems, but is limited by its computational complexity. We address this problem by presenting a novel algorithm that automatically computes an explicit approximation to nonlinear MPC schemes while retaining closed-loop guarantees. Specifically, the problem can be reduced to a function approximation problem, which we then tackle by proposing ALKIA-X, the Adaptive and Localized Kernel Interpolation Algorithm with eXtrapolated reproducing kernel Hilbert space norm. ALKIA-X is a non-iterative algorithm that ensures numerically well-conditioned computations, a fast-to-evaluate approximating function, and the guaranteed satisfaction of any desired bound on the approximation error. Hence, ALKIA-X automatically computes an explicit function that approximates the MPC, yielding a controller suitable for safety-critical systems and high sampling rates. We apply ALKIA-X to approximate two nonlinear MPC schemes, demonstrating reduced computational demand and applicability to realistic problems.
This paper studies the optimal control problem for discrete-time nonlinear systems and an approximate dynamic programming-based Model Predictive Control (MPC) scheme is proposed for minimizing a quadratic performance measure. In the proposed approach, the value function is approximated as a quadratic function for which the parametric matrix is computed using a switched system approximate of the nonlinear system. The approach is modified further using a multi-stage scheme to improve the control accuracy and an extension to incorporate state constraints. The MPC scheme is validated experimentally on a multi-tank system which is modeled as a third-order nonlinear system. The experimental results show the proposed MPC scheme results in significantly lesser online computation compared to the Nonlinear MPC scheme.