A hybrid transmitting and reflecting beyond diagonal reconfigurable intelligent surface (BD-RIS) design is proposed. Operating in the same aperture, frequency band and polarization, the proposed BD-RIS features independent beam steering control of its reflected and transmitted waves. In addition it provides a hybrid mode with both reflected and transmitted waves using tunable power splitting between beams. The BD-RIS comprises two phase reconfigurable antenna arrays interconnected by an array of tunable two-port power splitters. The two-port power splitter in each BD-RIS cell is built upon a varactor in parallel with a bias inductor to exert tunable impedance variations on transmission lines. Provided with variable reverse DC voltages, the two-port power splitter can control the power ratio of S11 over S21 from -20 dB to 20 dB, thus allowing tunable power splitting. Each antenna is 2-bit phase reconfigurable with 200 MHz bandwidth at 2.4 GHz so that each cell of BD-RIS can also achieve independent reflection and transmission phase control. To characterize and optimize the electromagnetic response of the proposed BD-RIS design, a Th\'evenin equivalent model and corresponding analytical method is provided. A BD-RIS with 4 by 4 cells was also prototyped and tested. Experiments show that in reflection and transmission mode, the fabricated BD-RIS can realize beam steering in reflection and transmission space, respectively. It is also verified that when operating in hybrid mode, the BD-RIS enables independent beam steering of the reflected and transmitted waves. This work helps fill the gap between realizing practical hardware design and establishing an accurate physical model for the hybrid transmitting and reflecting BD-RIS, enabling hybrid transmitting and reflecting BD-RIS assisted wireless communications.
Teleoperated driving (ToD) systems are a special type of cyber-physical system (CPS) where the operator remotely controls the steering, acceleration, and braking actions of the vehicle. Malicious actors may inject false data into communication channels to manipulate the teleoperator's driving commands to cause harm. Hence, protection of this communication is necessary for a safe operation of the target vehicle. However, according to the National Institute of Standards and Technology (NIST) cybersecurity framework, protection is not enough, and detecting an attack is necessary. Moreover, UN R155 mandates that vehicle fleets detect and log security incidents. Thus, the cyber-physical threats of ToD are modeled using the attack-centric approach in this paper. Then, an attack model with false data injection (FDI) on the steering control command is created from real vehicle data. A risk of this attack model is assessed for a last-mile delivery (LMD) application. Finally, a physics-based context-aware anomaly detection system (PCADS) is proposed to detect such false injection attacks, and preliminary experimental results are presented to validate the model.
This study introduces the Perception Latency Mitigation Network (PLM-Net), a novel deep learning approach for addressing perception latency in vision-based Autonomous Vehicle (AV) lateral control systems. Perception latency is the delay between capturing the environment through vision sensors (e.g., cameras) and applying an action (e.g., steering). This issue is understudied in both classical and neural-network-based control methods. Reducing this latency with powerful GPUs and FPGAs is possible but impractical for automotive platforms. PLM-Net comprises the Base Model (BM) and the Timed Action Prediction Model (TAPM). BM represents the original Lane Keeping Assist (LKA) system, while TAPM predicts future actions for different latency values. By integrating these models, PLM-Net mitigates perception latency. The final output is determined through linear interpolation of BM and TAPM outputs based on real-time latency. This design addresses both constant and varying latency, improving driving trajectories and steering control. Experimental results validate the efficacy of PLM-Net across various latency conditions. Source code: https://github.com/AwsKhalil/oscar/tree/devel-plm-net.
With the development of autonomous driving technology, there are increasing demands for vehicle control, and MPC has become a widely researched topic in both industry and academia. Existing MPC control methods based on vehicle kinematics or dynamics have challenges such as difficult modeling, numerous parameters, strong nonlinearity, and high computational cost. To address these issues, this paper adapts an existing Data-driven MPC control method and applies it to autonomous vehicle steering control. This method avoids the need for complex vehicle system modeling and achieves trajectory tracking with relatively low computational time and small errors. We validate the control effectiveness of the algorithm in specific scenario through CarSim-Simulink simulation and perform comparative analysis with PID and vehicle kinematics MPC, confirming the feasibility and superiority of it for vehicle steering control.
The technology of autonomous driving is currently attracting a great deal of interest in both research and industry. In this paper, we present a deep learning dual-model solution that uses two deep neural networks for combined braking and steering in autonomous vehicles. Steering control is achieved by applying the NVIDIA's PilotNet model to predict the steering wheel angle, while braking control relies on the use of MobileNet SSD. Both models rely on a single front-facing camera for image input. The MobileNet SSD model is suitable for devices with constrained resources, whereas PilotNet struggles to operate efficiently on smaller devices with limited resources. To make it suitable for such devices, we modified the PilotNet model using our own original network design and reduced the number of model parameters and its memory footprint by approximately 60%. The inference latency has also been reduced, making the model more suitable to operate on resource-constrained devices. The modified PilotNet model achieves similar loss and accuracy compared to the original PilotNet model. When evaluated in a simulated environment, both autonomous driving systems, one using the modified PilotNet model and the other using the original PilotNet model for steering, show similar levels of autonomous driving performance.
A ship steering control is designed for a nonlinear maneuvering model whose rudder manipulation is constrained in both magnitude and rate. In our method, the tracking problem of the target heading angle with input constraints is converted into the tracking problem for a strict-feedback system without any input constraints. To derive this system, hyperbolic tangent ($\tanh$) function and auxiliary variables are introduced to deal with the input constraints. Furthermore, using the feature of the derivative of $\tanh$ function, auxiliary systems are successfully derived in the strict-feedback form. The backstepping method is utilized to construct the feedback control law for the resulting cascade system. The proposed steering control is verified in numerical experiments, and the result shows that the tracking of the target heading angle is successful using the proposed control law.
Autonomous vehicles are the upcoming solution to most transportation problems such as safety, comfort, and efficiency. The steering control is one of the main important tasks in achieving autonomous driving. Model predictive control (MPC) is among the fittest controllers for this task due to its optimal performance and ability to handle constraints. This paper proposes an adaptive MPC controller (AMPC) for the path-tracking task and an improved PSO algorithm for optimizing the AMPC parameters. Parameter adaption is realized online using a lookup table approach. The proposed AMPC performance is assessed and compared with the classic MPC and the Pure Pursuit controller through simulations.
Vehicle manufacturers are racing to create autonomous navigation and steering control algorithms for their vehicles. These software are made to handle various real-life scenarios such as obstacle avoidance and lane maneuvering. There is some ongoing research to incorporate pothole avoidance into these autonomous systems. However, there is very little research on the effect of hitting a pothole on the autonomous navigation software that uses cameras to make driving decisions. Perturbations in the camera angle when hitting a pothole can cause errors in the predicted steering angle. In this paper, we present a new model to compensate for such angle perturbations and reduce any errors in steering control prediction algorithms. We evaluate our model on perturbations of publicly available datasets and show our model can reduce the errors in the estimated steering angle from perturbed images to 2.3%, making autonomous steering control robust against the dash cam image angle perturbations induced when one wheel of a car goes over a pothole.
The steering control of an autonomous unicycle is considered. The underlying dynamical model of a single rolling wheel is discussed regarding the steady state motions and their stability. The unicycle model is introduced as the simplest possible extension of the rolling wheel where the location of the center of gravity is controlled. With the help of the Appellian approach, a state space representation of the controlled nonholonomic system is built in a way that the most compact nonlinear equations of motions are constructed. Based on controllability analysis, feedback controllers are designed which successfully carry out lane changing and turning maneuvers. The behavior of the closed-loop system is demonstrated by numerical simulations.
Unexpected yaw disturbances like braking on unilaterally icy road, side wind forces and tire rupture are very difficult to handle by the driver of a road vehicle, due to his/her large panic reaction period ranging between 0.5 to 2 seconds. Automatic driver assist systems provide counteracting yaw moments during this driver panic reaction period to maintain the stability of the yaw dynamics of the vehicle. An active steering based driver assist system that uses the model regulator control architecture is introduced and used here for yaw dynamics stabilization in such situations. The model regulator which is a special form of a two degree of freedom control architecture is introduced and explained in detail in a tutorial fashion whereby its integral action capability, among others, is also shown. An auxiliary steering actuation system is assumed and a limited integrator version of the model regulator based steering controller is developed in order not to saturate the auxiliary steering actuator. This low frequency limited integrator implementation also allows the driver to take care of low frequency steering and disturbance rejection tasks. Linear simulation results are used to demonstrate the effectiveness of the proposed method.
This article presents the design and real-time implementation of an optimal controller for precise steering control of a remotely operated underwater vehicle (ROV). A PI controller is investigated to achieve the desired steering performance. The gain parameters of the controller are tuned using the genetic algorithm (GA). The experimental response corresponding to the step waveform for the GA is obtained. A root-locus-tuned PI controller alongside a simulated-annealing-based PI controller (SAPI) is used to benchmark the response characteristics such as overshoot, peak time, and settling time. The experimental findings indicate that GAPI provides considerably better performance than SAPI and the root-locus-tuned controller.
This paper studies the problem of steering the distribution of a linear time-invariant system from an initial normal distribution to a terminal normal distribution under no knowledge of the system dynamics. This data-driven control framework uses data collected from the input and the state and utilizes the seminal work by Willems et al. to construct a data-based parametrization of the mean and the covariance control problems. These problems are then solved to optimality as convex programs using standard techniques from the covariance control literature. We also discuss the equivalence of indirect and direct data-driven covariance steering designs, as well as a regularized version of the problem that provides a balance between the two. We illustrate the proposed framework through a set of randomized trials on a double integrator system and show that the results match up almost exactly with the corresponding model-based method in the noiseless case. We then analyze the robustness properties of the data-free and data-driven covariance steering methods and demonstrate the trade-offs between performance and optimality among these methods in the presence of data corrupted with exogenous noise.
Wind farm modelling has been an area of rapidly increasing interest with numerous analytical as well as computational-based approaches developed to extend the margins of wind farm efficiency and maximise power production. In this work, we present the novel ML framework WakeNet, which can reproduce generalised 2D turbine wake velocity fields at hub-height over a wide range of yaw angles, wind speeds and turbulence intensities (TIs), with a mean accuracy of 99.8% compared to the solution calculated using the state-of-the-art wind farm modelling software FLORIS. As the generation of sufficient high-fidelity data for network training purposes can be cost-prohibitive, the utility of multi-fidelity transfer learning has also been investigated. Specifically, a network pre-trained on the low-fidelity Gaussian wake model is fine-tuned in order to obtain accurate wake results for the mid-fidelity Curl wake model. The robustness and overall performance of WakeNet on various wake steering control and layout optimisation scenarios has been validated through power-gain heatmaps, obtaining at least 90% of the power gained through optimisation performed with FLORIS directly. We also demonstrate that when utilising the Curl model, WakeNet is able to provide similar power gains to FLORIS, two orders of magnitude faster (e.g. 10 minutes vs 36 hours per optimisation case). The wake evaluation time of wakeNet when trained on a high-fidelity CFD dataset is expected to be similar, thus further increasing computational time gains. These promising results show that generalised wake modelling with ML tools can be accurate enough to contribute towards active yaw and layout optimisation, while producing realistic optimised configurations at a fraction of the computational cost, hence making it feasible to perform real-time active yaw control as well as robust optimisation under uncertainty.
With the recent emergence of deep learning, computer vision-based applications have demonstrated better applicability in accomplishing driving tasks including drivable road region detection, lane keeping and steering control in self-driving cars. Till recently, numerous lane-marking detection based steering control and lane keeping methods have been proposed to perform autonomous driving on urban well-structured roads. But the matter of fact is that these methods are not feasible on roads where lane markings are not available or faded over time which makes drivable road region detection a crucial task. Moreover, it is highly arduous task to estimate steering angle on deteriorated roads using existing road detection and steering angle estimation methods. To the best of our knowledge, there is no standard benchmark available for drivable road region detection and steering angle estimation on unstructured roads. To this end, we present a large-scale dataset for drivable region road detection, comprising of 15,000-pixel level high quality fine annotations. Alongside dataset, we also present an end-to-end drivable road region detection and steering angle estimation method to ensure the autonomous driving on generalized urban, rural, and unstructured road conditions. The proposed method performs pixel-level segmentation to extract drivable road region and quantifies lane interception to estimate the steering angle of self-driving cars. A comprehensive qualitative and quantitative analysis has been carried out to demonstrate the effectiveness of our proposed dataset, road detection and steering angle estimation methods. Our benchmark is available at https://carl-dataset.github.io/index/ .
Nonlinear Robust Model Predictive Control (RMPC) provides a very promising solution to the problem of automatic emergency maneuvering, which is capable of handling multiple possibly conflicting objectives of robustness and performance. Even though RMPC gives a suboptimal solution, the key challenge in real-time implementation is that it is computationally very demanding. In this paper a real-time capable robust tube MPC based framework for steering control during emergency obstacle avoidance maneuver is presented. The novelty of this framework lies in the robust integration of path planning and path following tasks of autonomous vehicles. A simulation study showcases the robust performance improvements due to the proposed strategy over a non-robust MPC in different extreme maneuvering scenarios.
Driver distraction is a well-known cause for traffic collisions worldwide. Studies have indicated that shared steering control, which actively provides haptic guidance torque on the steering wheel, effectively improves the performance of distracted drivers. Recently, adaptive shared steering control based on the physiological status of the driver has been developed, although its effect on distracted driver behavior remains unclear. To this end, a high-fidelity driving simulator experiment was conducted involving 18 participants performing double lane changes. The experimental conditions comprised two driver states: attentive and distracted. Under each condition, evaluations were performed on three types of haptic guidance: none (manual), fixed authority, and adaptive authority based on feedback from the forearm surface electromyography of the driver. Evaluation results indicated that, for both attentive and distracted drivers, haptic guidance with adaptive authority yielded lower driver workload and reduced lane departure risk than manual driving and fixed authority. Moreover, there was a tendency for distracted drivers to reduce grip strength on the steering wheel to follow the haptic guidance with fixed authority, resulting in a relatively shorter double lane change duration.
In this paper, we present a novel end-to-end deep neural network model for autonomous driving that takes monocular image sequence as input, and directly generates the steering control angle. Firstly, we model the end-to-end driving problem as a local path planning process. Inspired by the environmental representation in the classical planning algorithms(i.e. the beam curvature method), pixel-wise orientations are fed into the network to learn direction-aware features. Next, to handle the imbalanced distribution of steering values in training datasets, we propose an improvement on a cost-sensitive loss function named SteeringLoss2. Besides, we also present a new end-to-end driving dataset, which provides corresponding LiDAR and image sequences, as well as standard driving behaviors. Our dataset includes multiple driving scenarios, such as urban, country, and off-road. Numerous experiments are conducted on both public available LiVi-Set and our own dataset, and the results show that the model using our proposed methods can predict steering angle accurately.
Haptic guidance in a shared steering assistance system has drawn significant attention in intelligent vehicle fields, owing to its mutual communication ability for vehicle control. By exerting continuous torque on the steering wheel, both the driver and support system can share lateral control of the vehicle. However, current haptic guidance steering systems demonstrate some deficiencies in assisting lane changing. This study explored a new steering interaction method, including the design and evaluation of an intention-based haptic shared steering system. Such an intention-based method can support both lane keeping and lane changing assistance, by detecting a driver lane change intention. By using a deep learning-based method to model a driver decision timing regarding lane crossing, an adaptive gain control method was proposed for realizing a steering control system. An intention consistency method was proposed to detect whether the driver and the system were acting towards the same target trajectories and to accurately capture the driver intention. A driving simulator experiment was conducted to test the system performance. Participants were required to perform six trials with assistive methods and one trial without assistance. The results demonstrated that the supporting system decreased the lane departure risk in the lane keeping tasks and could support a fast and stable lane changing maneuver.
Hand-crafting generalised decision-making rules for real-world urban autonomous driving is hard. Alternatively, learning behaviour from easy-to-collect human driving demonstrations is appealing. Prior work has studied imitation learning (IL) for autonomous driving with a number of limitations. Examples include only performing lane-following rather than following a user-defined route, only using a single camera view or heavily cropped frames lacking state observability, only lateral (steering) control, but not longitudinal (speed) control and a lack of interaction with traffic. Importantly, the majority of such systems have been primarily evaluated in simulation - a simple domain, which lacks real-world complexities. Motivated by these challenges, we focus on learning representations of semantics, geometry and motion with computer vision for IL from human driving demonstrations. As our main contribution, we present an end-to-end conditional imitation learning approach, combining both lateral and longitudinal control on a real vehicle for following urban routes with simple traffic. We address inherent dataset bias by data balancing, training our final policy on approximately 30 hours of demonstrations gathered over six months. We evaluate our method on an autonomous vehicle by driving 35km of novel routes in European urban streets.
We present a training pipeline for the autonomous driving task given the current camera image and vehicle speed as the input to produce the throttle, brake, and steering control output. The simulator Airsim's convenient weather and lighting API provides a sufficient diversity during training which can be very helpful to increase the trained policy's robustness. In order to not limit the possible policy's performance, we use a continuous and deterministic control policy setting. We utilize ResNet-34 as our actor and critic networks with some slight changes in the fully connected layers. Considering human's mastery of this task and the high-complexity nature of this task, we first use imitation learning to mimic the given human policy and leverage the trained policy and its weights to the reinforcement learning phase for which we use DDPG. This combination shows a considerable performance boost comparing to both pure imitation learning and pure DDPG for the autonomous driving task.