Autonomous mobility-on-demand systems are a viable alternative to mitigate many transportation-related externalities in cities, such as rising vehicle volumes in urban areas and transportation-related pollution. However, the success of these systems heavily depends on efficient and effective fleet control strategies. In this context, we study online control algorithms for autonomous mobility-on-demand systems and develop a novel hybrid combinatorial optimization enriched machine learning pipeline which learns online dispatching and rebalancing policies from optimal full-information solutions. We test our hybrid pipeline on large-scale real-world scenarios with different vehicle fleet sizes and various request densities. We show that our approach outperforms state-of-the-art greedy, and model-predictive control approaches with respect to various KPIs, e.g., by up to 17.1% and on average by 6.3% in terms of realized profit.
Biotechnology offers many opportunities for the sustainable manufacturing of valuable products. The toolbox to optimize bioprocesses includes \textit{extracellular} process elements such as the bioreactor design and mode of operation, medium formulation, culture conditions, feeding rates, etc. However, these elements are frequently insufficient for achieving optimal process performance or precise product composition. One can use metabolic and genetic engineering methods for optimization at the intracellular level. Nevertheless, those are often of static nature, failing when applied to dynamic processes or if disturbances occur. Furthermore, many bioprocesses are optimized empirically and implemented with little-to-no feedback control to counteract disturbances. The concept of cybergenetics has opened new possibilities to optimize bioprocesses by enabling online modulation of the gene expression of metabolism-relevant proteins via external inputs (e.g., light intensity in optogenetics). Here, we fuse cybergenetics with model-based optimization and predictive control for optimizing dynamic bioprocesses. To do so, we propose to use dynamic constraint-based models that integrate the dynamics of metabolic reactions, resource allocation, and inducible gene expression. We formulate a model-based optimal control problem to find the optimal process inputs. Furthermore, we propose using model predictive control to address uncertainties via online feedback. We focus on fed-batch processes, where the substrate feeding rate is an additional optimization variable. As a simulation example, we show the optogenetic control of the ATPase enzyme complex for dynamic modulation of enforced ATP wasting to adjust product yield and productivity.
The ability to accurately predict others' behavior is central to the safety and efficiency of interactive robotics. Unfortunately, robots often lack access to key information on which these predictions may hinge, such as other agents' goals, attention, and willingness to cooperate. Dual control theory addresses this challenge by treating unknown parameters of a predictive model as stochastic hidden states and inferring their values at runtime using information gathered during system operation. While able to optimally and automatically trade off exploration and exploitation, dual control is computationally intractable for general interactive motion planning. In this paper, we present a novel algorithmic approach to enable active uncertainty reduction for interactive motion planning based on the implicit dual control paradigm. Our approach relies on sampling-based approximation of stochastic dynamic programming, leading to a model predictive control problem that can be readily solved by real-time gradient-based optimization methods. The resulting policy is shown to preserve the dual control effect for a broad class of predictive models with both continuous and categorical uncertainty. To ensure the safe operation of the interacting agents, we use a runtime safety filter (also referred to as a "shielding" scheme), which overrides the robot's dual control policy with a safety fallback strategy when a safety-critical event is imminent. We then augment the dual control framework with an improved variant of the recently proposed shielding-aware robust planning scheme, which proactively balances the nominal planning performance with the risk of high-cost emergency maneuvers triggered by low-probability agent behaviors. We demonstrate the efficacy of our approach with both simulated driving studies and hardware experiments using 1/10 scale autonomous vehicles.
Model Predictive Control (MPC) is attracting tremendous attention in the autonomous driving task as a powerful control technique. The success of an MPC controller strongly depends on an accurate internal dynamics model. However, the static parameters, usually learned by system identification, often fail to adapt to both internal and external perturbations in real-world scenarios. In this paper, we firstly (1) reformulate the problem as a Partially Observed Markov Decision Process (POMDP) that absorbs the uncertainties into observations and maintains Markov property into hidden states; and (2) learn a recurrent policy continually adapting the parameters of the dynamics model via Recurrent Reinforcement Learning (RRL) for optimal and adaptive control; and (3) finally evaluate the proposed algorithm (referred as $\textit{MPC-RRL}$) in CARLA simulator and leading to robust behaviours under a wide range of perturbations.
We present a data-driven modeling and control framework for physics-based building emulators. Our approach consists of: (a) Offline training of differentiable surrogate models that accelerate model evaluations, provide cost-effective gradients, and maintain good predictive accuracy for the receding horizon in Model Predictive Control (MPC), and (b) Formulating and solving nonlinear building HVAC MPC problems. We extensively evaluate the modeling and control performance using multiple surrogate models and optimization frameworks across various test cases available in the Building Optimization Testing Framework (BOPTEST). Our framework is compatible with other modeling techniques and can be customized with different control formulations, making it adaptable and future-proof for test cases currently under development for BOPTEST. This modularity provides a path towards prototyping predictive controllers in large buildings, ensuring scalability and robustness in real-world applications.
This paper proposes a set of technological solutions to transform existing transport systems into more intelligent, interactive systems by utilizing optimization and control methods that can be implemented in the near future. This will result in improved public services and quality of life for residents. Three application scenes that are closely related to people's daily life are discussed. We first propose a traffic light scheduling strategy using a model predictive control (MPC) method, with the aim of fairly minimizing delays for both pedestrians and vehicles. Then, a combined dispatching-operation system is proposed to increase control flexibility, with a corresponding implementation solution for boarding control. Finally, a possible scheme to combine both public transport and autonomous vehicle systems is proposed to improve existing public transport systems.
Cellular networks are becoming increasingly heterogeneous with higher base station (BS) densities and ever more frequency bands, making BS selection and band assignment key decisions in terms of rate and coverage. In this paper, we decompose the mobility-aware user association task into (i) forecasting of user rate and then (ii) convex utility maximization for user association accounting for the effects of BS load and handover overheads. Using a linear combination of normalized mean-squared error and normalized discounted cumulative gain as a novel loss function, a recurrent deep neural network is trained to reliably forecast the mobile users' future rates. Based on the forecast, the controller optimizes the association decisions to maximize the service rate-based network utility using our computationally efficient (speed up of 100x versus generic convex solver) algorithm based on the Frank-Wolfe method. Using an industry-grade network simulator developed by Meta, we show that the proposed model predictive control (MPC) approach improves the 5th percentile service rate by 3.5x compared to the traditional signal strength-based association, reduces the median number of handovers by 7x compared to a handover agnostic strategy, and achieves service rates close to a genie-aided scheme. Furthermore, our model-based approach is significantly more sample-efficient (needs 100x less training data) compared to model-free reinforcement learning (RL), and generalizes well across different user drop scenarios.
Distributed model predictive control (DMPC) is a flexible and scalable feedback control method applicable to a wide range of systems. While the stability analysis of DMPC is quite well understood, there exist only limited implementation results for realistic applications involving distributed computation and networked communication. This article approaches formation control of mobile robots via a cooperative DMPC scheme. We discuss the implementation via decentralized optimization algorithms. To this end, we combine the alternating direction method of multipliers with decentralized sequential quadratic programming to solve the underlying optimal control problem in a decentralized fashion with nominal convergence guarantees. Our approach only requires coupled subsystems to communicate and does not rely on a central coordinator. Our experimental results showcase the efficacy of DMPC for formation control and they demonstrate the real-time feasibility of the considered algorithms.
Full drive-by-wire electric vehicles (FDWEV) with X-by-wire technology can achieve independent driving, braking, and steering of each wheel, providing a good application platform for autonomous driving technology. Path planning and tracking control, in particular, are critical components of autonomous driving. However, It is challenging to comprehensively design an robust control algorithm by integrating vehicle path planning in a complicated unstructured scenario for FDWEV. To address the above issue, this paper first proposes the artificial potential field (APF) method for path planning in the prescribed park with different static obstacles to generate the reference path information, where speed planning is incorporated considering kinematics and dynamic constraints. Second, two tracking control methods, curvature calculation (CC-based) and model predictive control (MPC-based) methods with the lateral dynamics model, are proposed to track the desired path under different driving conditions, in which a forward-looking behavior model of the driver with variable preview distance is designed based on fuzzy control theory. CarSim-AMESim-Simulink co-simulation is conducted with the existence of obstacles. The simulation results show that the proposed two control approaches are effective for many driving scenarios and the MPC-based path-tracking controller enhances dynamic tracking performance and ensures good maneuverability under high-dynamic driving conditions.
In future energy systems with high shares of renewable energy sources, the electricity demand of buildings has to react to the fluctuating electricity generation in view of stability. As buildings consume one-third of global energy and almost half of this energy accounts for Heating, Ventilation, and Air Conditioning (HVAC) systems, HVAC are suitable for shifting their electricity consumption in time. To this end, intelligent control strategies are necessary as the conventional control of HVAC is not optimized for the actual demand of occupants and the current situation in the electricity grid. In this paper, we present the novel multi-zone controller Price Storage Control (PSC) that not only considers room-individual Occupants' Thermal Satisfaction (OTS), but also the available energy storage, and energy prices. The main feature of PSC is that it does not need a building model or forecasts of future demands to derive the control actions for multiple rooms in a building. For comparison, we use an ideal, error-free Model Predictive Control (MPC), a simplified variant without storage consideration (PC), and a conventional hysteresis-based two-point control. We evaluate the four controllers in a multi-zone environment for heating a building in winter and consider two different scenarios that differ in how much the permitted temperatures vary. In addition, we compare the impact of model parameters with high and low thermal capacitance. The results show that PSC strongly outperforms the conventional control approach in both scenarios and for both parameters. For high capacitance, it leads to 22 % costs reduction while the ideal MPC achieves cost reductions of more than 39 %. Considering that PSC does not need any building model or forecast, as opposed to MPC, the results support the suitability of our developed control strategy for controlling HVAC systems in future energy systems.
This paper presents an approach for learning Model Predictive Control (MPC) schemes directly from data using Reinforcement Learning (RL) methods. The state-of-the-art learning methods use RL to improve the performance of parameterized MPC schemes. However, these learning algorithms are often gradient-based methods that require frequent evaluations of computationally expensive MPC schemes, thereby restricting their use on big datasets. We propose to tackle this issue by using tools from RL to learn a parameterized MPC scheme directly from data in an offline fashion. Our approach derives an MPC scheme without having to solve it over the collected dataset, thereby eliminating the computational complexity of existing techniques for big data. We evaluate the proposed method on three simulated experiments of varying complexity.
This paper presents an online-capable controller for the energy management system of a parallel hybrid electric vehicle based on model predictive control. Its task is to minimize the vehicle's fuel consumption along a predicted driving mission by calculating the distribution of the driver's power request between the electrical and the combustive part of the powertrain, and by choosing the driving mode, which depends on the vehicle's clutch state. The inclusion of the clutch state in a model predictive control structure is not trivial because the underlying optimization problem becomes a mixed-integer program as a consequence. Using Pontryagin's Minimum Principle and a simplified vehicle model, it is possible to prove that a drive cycle-dependent critical power request Pcrit exists, which uniquely separates the optimal driving mode. Based on this result, a learning algorithm is proposed to determine Pcrit during the operation of the vehicle. The learning algorithm is incorporated into a multi-level controller structure and the working principle of the resulting multi-level learning-based model predictive controller is analyzed in detail using two realistic driving missions. A comparison to the solution obtained by Dynamic Programming reveals that the proposed controller achieves close-to-optimal performance.
Trains are a corner stone of public transport and play an important role in daily life. A challenging task in train operation is to avoid skidding and sliding during fast changes of traction conditions, which can, for example, occur due to changing weather conditions, crossings, tunnels or forest entries. The latter depends on local track conditions and can be recorded in a map together with other location-dependent information like speed limits and inclination. In this paper, a model predictive control (MPC) approach is developed. Thanks to the knowledge of future changes of traction conditions, the approach is able to avoid short-term skidding and sliding even under fast changes of traction conditions. In a first step, an optimal reference trajectory is determined by a multiple-shooting approach. In a second step, the reference trajectory is tracked by an MPC setup. The developed method is simulated along a track with fast-changing traction conditions for different scenarios, like changing weather conditions and unexpected delays. In all cases, skidding and sliding is avoided.
Fuel cell electric vehicles have earned substantial attentions in recent decades due to their high-efficiency and zero-emission features, while the high operating costs remain the major barrier towards their large-scale commercialization. In such context, this paper aims to devise an energy management strategy for an urban postal-delivery fuel cell electric vehicle for operating cost mitigation. First, a data-driven dual-loop spatial-domain battery state-of-charge reference estimator is designed to guide battery energy depletion, which is trained by real-world driving data collected in postal delivery missions. Then, a fuzzy C-means clustering enhanced Markov speed predictor is constructed to project the upcoming velocity. Lastly, combining the state-of-charge reference and the forecasted speed, a model predictive control-based cost-optimization energy management strategy is established to mitigate vehicle operating costs imposed by energy consumption and power-source degradations. Validation results have shown that 1) the proposed strategy could mitigate the operating cost by 4.43% and 7.30% in average versus benchmark strategies, denoting its superiority in term of cost-reduction and 2) the computation burden per step of the proposed strategy is averaged at 0.123ms, less than the sampling time interval 1s, proving its potential of real-time applications.
In recent years, nonlinear model predictive control (NMPC) has been extensively used for solving automotive motion control and planning tasks. In order to formulate the NMPC problem, different coordinate systems can be used with different advantages. We propose and compare formulations for the NMPC related optimization problem, involving a Cartesian and a Frenet coordinate frame (CCF/ FCF) in a single nonlinear program (NLP). We specify costs and collision avoidance constraints in the more advantageous coordinate frame, derive appropriate formulations and compare different obstacle constraints. With this approach, we exploit the simpler formulation of opponent vehicle constraints in the CCF, as well as road aligned costs and constraints related to the FCF. Comparisons to other approaches in a simulation framework highlight the advantages of the proposed approaches.
Heating in private households is a major contributor to the emissions generated today. Heat pumps are a promising alternative for heat generation and are a key technology in achieving our goals of the German energy transformation and to become less dependent on fossil fuels. Today, the majority of heat pumps in the field are controlled by a simple heating curve, which is a naive mapping of the current outdoor temperature to a control action. A more advanced control approach is model predictive control (MPC) which was applied in multiple research works to heat pump control. However, MPC is heavily dependent on the building model, which has several disadvantages. Motivated by this and by recent breakthroughs in the field, this work applies deep reinforcement learning (DRL) to heat pump control in a simulated environment. Through a comparison to MPC, it could be shown that it is possible to apply DRL in a model-free manner to achieve MPC-like performance. This work extends other works which have already applied DRL to building heating operation by performing an in-depth analysis of the learned control strategies and by giving a detailed comparison of the two state-of-the-art control methods.
We present a numerically efficient Nonlinear Model Predictive Control (NMPC) approach, called Set Membership based NMPC (SM-NMPC). In particular, a Set Membership method is used to derive from data an approximation and tight bounds on the optimal NMPC control law. These quantities are used to reduce the dimensionality and volume of the search domain of the NMPC optimization problem, allowing a significant shortening of the computation time. The proposed SM-NMPC strategy is tested in simulation, considering realistic autonomous vehicle scenarios, like parallel parking and lane keeping maneuvers.
This paper proposes an interaction and safety-aware motion-planning method for an autonomous vehicle in uncertain multi-vehicle traffic environments. The method integrates the ability of the interaction-aware interacting multiple model Kalman filter (IAIMM-KF) to predict interactive multi-modal maneuvers of surrounding vehicles, and the advantage of model predictive control (MPC) in planning an optimal trajectory in uncertain dynamic environments. The multi-modal prediction uncertainties, containing both the maneuver and trajectory uncertainties of surrounding vehicles, are considered in computing the reference targets and designing the collision-avoidance constraints of MPC for resilient motion planning of the ego vehicle. The MPC achieves safety awareness by incorporating a tunable parameter to adjust the predicted obstacle occupancy in the design of the safety constraints, allowing the approach to achieve a trade-off between performance and robustness. Based on the prediction of the surrounding vehicles, an optimal reference trajectory of the ego vehicle is computed by MPC to follow the time-varying reference targets and avoid collisions with obstacles. The efficiency of the method is illustrated in challenging highway-driving simulation scenarios and a driving scenario from a recorded traffic dataset.
This work studies the standoff tracking problem to drive an unmanned aerial vehicle (UAV) to slide on a desired circle over a moving target at a constant height. We propose a novel Lyapunov guidance vector (LGV) field with tunable convergence rates for the UAV's trajectory planning and a deep neural network (DNN)-based model predictive control (MPC) scheme to track the reference trajectory. Then, we show how to collect samples for training the DNN offline and design an integral module (IM) to refine the tracking performance of our DNN-based MPC. Moreover, the hardware-in-the-loop (HIL) simulation with an FPGA@200MHz demonstrates that our method is a valid alternative to embedded implementations of MPC for addressing complex systems and applications which is impossible for directly solving the MPC optimization problems.
In this work, we propose an approach to determine the dosages of antithyroid agents to treat hyperthyroid patients. Instead of relying on a trial-and-error approach as it is commonly done in clinical practice, we suggest to determine the dosages by means of a model predictive control (MPC) scheme. To this end, we first extend a mathematical model of the pituitary-thyroid feedback loop such that the intake of methimazole, a common antithyroid agent, can be considered. Second, based on the extended model, we develop an MPC scheme to determine suitable dosages. In numerical simulations, we consider scenarios in which (i) patients are affected by Graves' disease and take the medication orally and (ii) patients suffering from a life-threatening thyrotoxicosis, in which the medication is usually given intravenously. Our conceptual study suggests that determining the medication dosages by means of an MPC scheme could be a promising alternative to the currently applied trial-and-error approach.