State estimation for legged robots is challenging due to their highly dynamic motion and limitations imposed by sensor accuracy. By integrating Kalman filtering, optimization, and learning-based modalities, we propose a hybrid solution that combines proprioception and exteroceptive information for estimating the state of the robot's trunk. Leveraging joint encoder and IMU measurements, our Kalman filter is enhanced through a single-rigid body model that incorporates ground reaction force control outputs from convex Model Predictive Control optimization. The estimation is further refined through Gated Recurrent Units, which also considers semantic insights and robot height from a Vision Transformer autoencoder applied on depth images. This framework not only furnishes accurate robot state estimates, including uncertainty evaluations, but can minimize the nonlinear errors that arise from sensor measurements and model simplifications through learning. The proposed methodology is evaluated in hardware using a quadruped robot on various terrains, yielding a 65% improvement on the Root Mean Squared Error compared to our VIO SLAM baseline. Code example: https://github.com/AlexS28/OptiState
The emergence of novel the dummy data injection attack (DDIA) poses a severe threat to the secure and stable operation of power systems. These attacks are particularly perilous due to the minimal Euclidean spatial separation between the injected malicious data and legitimate data, rendering their precise detection challenging using conventional distance-based methods. Furthermore, existing research predominantly focuses on various machine learning techniques, often analyzing the temporal data sequences post-attack or relying solely on Euclidean spatial characteristics. Unfortunately, this approach tends to overlook the inherent topological correlations within the non-Euclidean spatial attributes of power grid data, consequently leading to diminished accuracy in attack localization. To address this issue, this study takes a comprehensive approach. Initially, it examines the underlying principles of these new DDIAs on power systems. Here, an intricate mathematical model of the DDIA is designed, accounting for incomplete topological knowledge and alternating current (AC) state estimation from an attacker's perspective. Subsequently, by integrating a priori knowledge of grid topology and considering the temporal correlations within measurement data and the topology-dependent attributes of the power grid, this study introduces temporal and spatial attention matrices. These matrices adaptively capture the spatio-temporal correlations within the attacks. Leveraging gated stacked causal convolution and graph wavelet sparse convolution, the study jointly extracts spatio-temporal DDIA features. Finally, the research proposes a DDIA localization method based on spatio-temporal graph neural networks. The accuracy and effectiveness of the DDIA model are rigorously demonstrated through comprehensive analytical cases.
This paper proposes an algebraic observer-based modulating function approach for linear time-variant systems and a class of nonlinear systems with discrete measurements. The underlying idea lies in constructing an observability transformation that infers some properties of the modulating function approach for designing such algebraic observers. First, we investigate the algebraic observer design for linear time-variant systems under an observable canonical form for continuous-time measurements. Then, we provide the convergence of the observation error in an L2-gain stability sense. Next, we develop an exponentially stable sampled-data observer which relies on the design of the algebraic observer and an output predictor to achieve state estimation from available measurements and under small inter-sampling periods. Using a trajectory-based approach, we prove the convergence of the observation error within a convergence rate that can be adjusted through the fixed time-horizon length of the modulating function and the upper bound of the sampling period. Furthermore, robustness of the sampled-data algebraic observer, which yields input-to-state stability, is inherited by the modulating kernel and the closed-loop output predictor design. Finally, we discuss the implementation procedure of the MF-based observer realization, demonstrate the applicability of the algebraic observer, and illustrate its performance through two examples given by linear time-invariant and linear time-variant systems with nonlinear input-output injection terms.
This paper proposes a privacy-preserving data fusion method for traffic state estimation (TSE). Unlike existing works that assume all data sources to be accessible by a single trusted party, we explicitly address data privacy concerns that arise in the collaboration and data sharing between multiple data owners, such as municipal authorities (MAs) and mobility providers (MPs). To this end, we propose a novel vertical federated learning (FL) approach, FedTSE, that enables multiple data owners to collaboratively train and apply a TSE model without having to exchange their private data. To enhance the applicability of the proposed FedTSE in common TSE scenarios with limited availability of ground-truth data, we further propose a privacy-preserving physics-informed FL approach, i.e., FedTSE-PI, that integrates traffic models into FL. Real-world data validation shows that the proposed methods can protect privacy while yielding similar accuracy to the oracle method without privacy considerations.
We investigate decentralized state estimation for a discrete event system in a setting where the information received at a coordinator may be corrupted or tampered by a malicious attacker. Specifically, a system is observed by a set of (local) observation sites (OSs) which occasionally send their recorded sequences of observations to the coordinator that is in charge of estimating the system state. The malfunctions and attacks, referred to as errors in this paper, include symbol deletions, insertions and replacements, each of which bears a positive cost. Two types of errors, global errors and local errors, are proposed to describe the impact of errors on decentralized information processing. Global errors occur when all OSs record the same error, while local errors occur when different OSs record different errors. Distinguishing these types of errors is important for a proper design of decentralized information processing (so as to be more resilient and better equipped to handle attacks and failures). For each type of error, we propose two methods to efficiently perform state estimation: one based on appropriately modifying the original system and the other based on inferring the matching behavior of the original system. For each method, we adopt an estimation-by-release methodology to design an algorithm for constructing a corresponding synchronizer for state estimation.
Traditional visual-inertial state estimation targets absolute camera poses and spatial landmark locations while first-order kinematics are typically resolved as an implicitly estimated sub-state. However, this poses a risk in velocity-based control scenarios, as the quality of the estimation of kinematics depends on the stability of absolute camera and landmark coordinates estimation. To address this issue, we propose a novel solution to tight visual-inertial fusion directly at the level of first-order kinematics by employing a dynamic vision sensor instead of a normal camera. More specifically, we leverage trifocal tensor geometry to establish an incidence relation that directly depends on events and camera velocity, and demonstrate how velocity estimates in highly dynamic situations can be obtained over short time intervals. Noise and outliers are dealt with using a nested two-layer RANSAC scheme. Additionally, smooth velocity signals are obtained from a tight fusion with pre-integrated inertial signals using a sliding window optimizer. Experiments on both simulated and real data demonstrate that the proposed tight event-inertial fusion leads to continuous and reliable velocity estimation in highly dynamic scenarios independently of absolute coordinates. Furthermore, in extreme cases, it achieves more stable and more accurate estimation of kinematics than traditional, point-position-based visual-inertial odometry.
Accurate estimation of the state of charge (SOC) and state of health (SOH) is crucial for the safe and reliable operation of batteries. Voltage measurement bias highly affects state estimation accuracy, especially in Lithium Iron Phosphate (LFP) batteries, which are susceptible due to their flat open-circuit voltage (OCV) curves. This work introduces a bias-compensated algorithm to reliably estimate the SOC and SOH of LFP batteries under the influence of voltage measurement bias. Specifically, SOC and SOH are estimated using the Dual Extended Kalman Filter (DEKF) in the high-slope SOC range, where voltage measurement bias effects are weak. Besides, the voltage measurement biases estimated in the low-slope SOC regions are compensated in the following joint estimation of SOC and SOH to enhance the state estimation accuracy further. Experimental results indicate that the proposed algorithm significantly outperforms the traditional method, which does not consider biases under different temperatures and aging conditions. Additionally, the bias-compensated algorithm can achieve low estimation errors of below 1.5% for SOC and 2% for SOH, even with a 30mV voltage measurement bias. Finally, even if the voltage measurement biases change in operation, the proposed algorithm can remain robust and keep the estimated errors of states around 2%.
We present a human state estimation framework that allows us to estimate the location, and even the activities, of people in an indoor environment without the requirement that they carry a specific devices with them. To achieve this "device free" localization we use a small number of low-cost Ultra-Wide Band (UWB) sensors distributed across the environment of interest. To achieve high quality estimation from the UWB signals merely reflected of people in the environment, we exploit a deep network that can learn to make inferences. The hardware setup consists of commercial off-the-shelf (COTS) single antenna UWB modules for sensing, paired with Raspberry PI units for computational processing and data transfer. We make use of the channel impulse response (CIR) measurements from the UWB sensors to estimate the human state - comprised of location and activity - in a given area. Additionally, we can also estimate the number of humans that occupy this region of interest. In our approach, first, we pre-process the CIR data which involves meticulous aggregation of measurements and extraction of key statistics. Afterwards, we leverage a convolutional deep neural network to map the CIRs into precise location estimates with sub-30 cm accuracy. Similarly, we achieve accurate human activity recognition and occupancy counting results. We show that we can quickly fine-tune our model for new out-of-distribution users, a process that requires only a few minutes of data and a few epochs of training. Our results show that UWB is a promising solution for adaptable smart-home localization and activity recognition problems.
This paper proposes a framework that combines online human state estimation, action recognition and motion prediction to enable early assessment and prevention of worker biomechanical risk during lifting tasks. The framework leverages the NIOSH index to perform online risk assessment, thus fitting real-time applications. In particular, the human state is retrieved via inverse kinematics/dynamics algorithms from wearable sensor data. Human action recognition and motion prediction are achieved by implementing an LSTM-based Guided Mixture of Experts architecture, which is trained offline and inferred online. With the recognized actions, a single lifting activity is divided into a series of continuous movements and the Revised NIOSH Lifting Equation can be applied for risk assessment. Moreover, the predicted motions enable anticipation of future risks. A haptic actuator, embedded in the wearable system, can alert the subject of potential risk, acting as an active prevention device. The performance of the proposed framework is validated by executing real lifting tasks, while the subject is equipped with the iFeel wearable system.
Mobile robotic systems are becoming increasingly popular. These systems are used in various indoor applications, raging from warehousing and manufacturing to test benches for assessment of advanced control strategies, such as artificial intelligence (AI)-based control solutions, just to name a few. Scaled robotic cars are commonly equipped with a hierarchical control acthiecture that includes tasks dedicated to vehicle state estimation and control. This paper covers both aspects by proposing (i) a federeted extended Kalman filter (FEKF), and (ii) a novel deep reinforcement learning (DRL) path tracking controller trained via an expert demonstrator to expedite the learning phase and increase robustess to the simulation-to-reality gap. The paper also presents the formulation of a vehicle model along with an effective yet simple procedure for identifying tis paramters. The experimentally validated model is used for (i) supporting the design of the FEKF and (ii) serving as a digital twin for training the proposed DRL-based path tracking algorithm. Experimental results confirm the ability of the FEKF to improve the estimate of the mobile robot's position. Furthermore, the effectiveness of the DRL path tracking strateguy is experimentally tested along manoeuvres not considered during training, showing also the ability of the AI-based solution to outpeform model-based control strategies and the demonstrator. The comparison with benchmraking controllers is quantitavely evalueted through a set of key performance indicators.
Unknown inputs related to, e.g., sensor aging, modeling errors, or device bias, represent a major concern in wireless sensor networks, as they degrade the state estimation performance. To improve the performance, unknown-input observers (UIOs) have been proposed. Most of the results available to design UIOs are based on explicit system models, which can be difficult or impossible to obtain in real-world applications. Data-driven techniques, on the other hand, have become a viable alternative for the design and analysis of unknown systems using only data. In this context, a novel data-driven distributed unknown-input observer (D-DUIO) for unknown continuous-time linear time-invariant (LTI) systems is developed, which requires solely some data collected offline, without any prior knowledge of the system matrices. In the paper, first, a model-based approach to the design of a DUIO is presented. A sufficient condition for the existence of such a DUIO is recalled, and a new one is proposed, that is prone to a data-driven adaption. Moving to a data-driven approach, it is shown that under suitable assumptions on the input/output/state data collected from the continuous-time system, it is possible to both claim the existence of a D-DUIO and to derive its matrices in terms of the matrices of pre-collected data. Finally, the efficacy of the D-DUIO is illustrated by means of numerical examples.
We use Koopman theory for data-driven model reduction of nonlinear dynamical systems with controls. We propose generic model structures combining delay-coordinate encoding of measurements and full-state decoding to integrate reduced Koopman modeling and state estimation. We present a deep-learning approach to train the proposed models. A case study demonstrates that our approach provides accurate control models and enables real-time capable nonlinear model predictive control of a high-purity cryogenic distillation column.
The applications of reference governors to systems with unmeasured set-bounded disturbances can lead to conservative solutions. This conservatism can be reduced by estimating the disturbance from output measurements and canceling it in the nominal control law. In this paper, a reference governor based on such an approach is considered and time-varying, disturbance and state estimation errors bounding sets are derived. Consequently, the traditional implementation of a reference governor, which exploits a constraint admissible positively-invariant set of constant commands and initial states, is replaced by one which utilizes a time-dependent sequence of similar sets (which are not necessary nested). Examples are reported which include two applications to longitudinal control of aircraft that illustrate handling of elevator uncertainty and wing icing.
State estimation is highly critical for accurately observing the dynamic behavior of the power grids and minimizing risks from cyber threats. However, existing state estimation methods encounter challenges in accurately capturing power system dynamics, primarily because of limitations in encoding the grid topology and sparse measurements. This paper proposes a physics-informed graphical learning state estimation method to address these limitations by leveraging both domain physical knowledge and a graph neural network (GNN). We employ a GNN architecture that can handle the graph-structured data of power systems more effectively than traditional data-driven methods. The physics-based knowledge is constructed from the branch current formulation, making the approach adaptable to both transmission and distribution systems. The validation results of three IEEE test systems show that the proposed method can achieve lower mean square error more than 20% than the conventional methods.
In this paper, we study state estimation for general nonlinear systems with unknown parameters and persistent process and measurement noise. In particular, we are interested in stability properties of the state estimate in the absence of persistency of excitation (PE). With a simple academic example, we show that existing moving horizon estimation (MHE) approaches as well as classical adaptive observers can result in diverging state estimates in the absence of PE, even if the noise is small. We propose a novel MHE formulation involving a regularization based on a constant prior estimate of the unknown system parameters. Only assuming the existence of a stable estimator, we prove that the proposed MHE results in practically robustly stable state estimates even in the absence of PE. We discuss the relation of the proposed MHE formulation to state-of-the-art results from MHE, adaptive estimation, and functional estimation. The properties of the proposed MHE approach are illustrated with a numerical example of a car with unknown tire friction parameters.
Due to the importance of satellites for society and the exponential increase in the number of objects in orbit, it is important to accurately determine the state (e.g., position and velocity) of these Resident Space Objects (RSOs) at any time and in a timely manner. State-of-the-art methodologies for initial orbit determination consist of Kalman-type filters that process sequential data over time and return the state and associated uncertainty of the object, as is the case of the Extended Kalman Filter (EKF). However, these methodologies are dependent on a good initial guess for the state vector and usually simplify the physical dynamical model, due to the difficulty of precisely modeling perturbative forces, such as atmospheric drag and solar radiation pressure. Other approaches do not require assumptions about the dynamical system, such as the trilateration method, and require simultaneous measurements, such as three measurements of range and range-rate for the particular case of trilateration. We consider the same setting of simultaneous measurements (one-shot), resorting to time delay and Doppler shift measurements. Based on recent advancements in the problem of moving target localization for sonar multistatic systems, we are able to formulate the problem of initial orbit determination as a Weighted Least Squares. With this approach, we are able to directly obtain the state of the object (position and velocity) and the associated covariance matrix from the Fisher's Information Matrix (FIM). We demonstrate that, for small noise, our estimator is able to attain the Cram\'er-Rao Lower Bound accuracy, i.e., the accuracy attained by the unbiased estimator with minimum variance. We also numerically demonstrate that our estimator is able to attain better accuracy on the state estimation than the trilateration method and returns a smaller uncertainty associated with the estimation.
Designing estimation algorithms for systems governed by partial differential equations (PDEs) such as fluid flows is challenging due to the high-dimensional and oftentimes nonlinear nature of the dynamics, as well as their dependence on unobserved physical parameters. In this paper, we propose two different lightweight and effective methodologies for real-time state estimation of PDEs in the presence of parametric uncertainties. Both approaches combine a Kalman filter with a data-driven polytopic linear reduced-order model obtained by dynamic mode decomposition (DMD). Using examples involving the nonlinear Burgers and Navier-Stokes equations, we demonstrate accurate estimation of both the state and the unknown physical parameter along system trajectories corresponding to various physical parameter values.
We consider two nonlinear state estimation problems in a setting where an extended Kalman filter receives measurements from two sets of sensors via two channels (2C). In the stochastic-2C problem, the channels drop measurements stochastically, whereas in 2C scheduling, the estimator chooses when to read each channel. In the first problem, we generalize linear-case 2C analysis to obtain -- for a given pair of channel arrival rates -- boundedness conditions for the trace of the error covariance, as well as a worst-case upper bound. For scheduling, an optimization problem is solved to find arrival rates that balance low channel usage with low trace bounds, and channels are read deterministically with the expected periods corresponding to these arrival rates. We validate both solutions in simulations for linear and nonlinear dynamics; as well as in a real experiment with an underwater robot whose position is being intermittently found in a UAV camera image.
Event cameras are bio-inspired, motion-activated sensors that demonstrate substantial potential in handling challenging situations, such as motion blur and high-dynamic range. In this paper, we proposed EVI-SAM to tackle the problem of 6 DoF pose tracking and 3D reconstruction using monocular event camera. A novel event-based hybrid tracking framework is designed to estimate the pose, leveraging the robustness of feature matching and the precision of direct alignment. Specifically, we develop an event-based 2D-2D alignment to construct the photometric constraint, and tightly integrate it with the event-based reprojection constraint. The mapping module recovers the dense and colorful depth of the scene through the image-guided event-based mapping method. Subsequently, the appearance, texture, and surface mesh of the 3D scene can be reconstructed by fusing the dense depth map from multiple viewpoints using truncated signed distance function (TSDF) fusion. To the best of our knowledge, this is the first non-learning work to realize event-based dense mapping. Numerical evaluations are performed on both publicly available and self-collected datasets, which qualitatively and quantitatively demonstrate the superior performance of our method. Our EVI-SAM effectively balances accuracy and robustness while maintaining computational efficiency, showcasing superior pose tracking and dense mapping performance in challenging scenarios. Video Demo: https://youtu.be/Nn40U4e5Si8.
This letter deals with the problem of state estimation for a class of systems involving linear dynamics with multiple quadratic output measurements. We propose a systematic approach to immerse the original system into a linear time-varying (LTV) system of a higher dimension. The methodology extends the original system by incorporating a minimum number of auxiliary states, ensuring that the resulting extended system exhibits both linear dynamics and linear output. Consequently, any Kalman-type observer can showcase global state estimation, provided the system is uniformly observable.