Humans' internal states play a key role in human-machine interaction, leading to the rise of human state estimation as a prominent field. Compared to swift state changes such as surprise and irritation, modeling gradual states like trust and satisfaction are further challenged by label sparsity: long time-series signals are usually associated with a single label, making it difficult to identify the critical span of state shifts. Windowing has been one widely-used technique to enable localized analysis of long time-series data. However, the performance of downstream models can be sensitive to the window size, and determining the optimal window size demands domain expertise and extensive search. To address this challenge, we propose a Selective Windowing Attention Network (SWAN), which employs window prompts and masked attention transformation to enable the selection of attended intervals with flexible lengths. We evaluate SWAN on the task of trust prediction on a new multimodal driving simulation dataset. Experiments show that SWAN significantly outperforms an existing empirical window selection baseline and neural network baselines including CNN-LSTM and Transformer. Furthermore, it shows robustness across a wide span of windowing ranges, compared to the traditional windowing approach.
In this article, the state estimation problems with unknown process noise and measurement noise covariances for both linear and nonlinear systems are considered. By formulating the joint estimation of system state and noise parameters into an optimization problem, a novel adaptive Kalman filter method based on conjugate-computation variational inference, referred to as CVIAKF, is proposed to approximate the joint posterior probability density function of the latent variables. Unlike the existing adaptive Kalman filter methods utilizing variational inference in natural-parameter space, CVIAKF performs optimization in expectation-parameter space, resulting in a faster and simpler solution. Meanwhile, CVIAKF divides optimization objectives into conjugate and non-conjugate parts of nonlinear dynamical models, whereas conjugate computations and stochastic mirror-descent are applied, respectively. Remarkably, the reparameterization trick is used to reduce the variance of stochastic gradients of the non-conjugate parts. The effectiveness of CVIAKF is validated through synthetic and real-world datasets of maneuvering target tracking.
This paper addresses the challenge of a particular class of noisy state observations in Markov Decision Processes (MDPs), a common issue in various real-world applications. We focus on modeling this uncertainty through a confusion matrix that captures the probabilities of misidentifying the true state. Our primary goal is to estimate the inherent measurement noise, and to this end, we propose two novel algorithmic approaches. The first, the method of second-order repetitive actions, is designed for efficient noise estimation within a finite time window, providing identifiable conditions for system analysis. The second approach comprises a family of Bayesian algorithms, which we thoroughly analyze and compare in terms of performance and limitations. We substantiate our theoretical findings with simulations, demonstrating the effectiveness of our methods in different scenarios, particularly highlighting their behavior in environments with varying stationary distributions. Our work advances the understanding of reinforcement learning in noisy environments, offering robust techniques for more accurate state estimation in MDPs.
This paper addresses the problem of data-driven modeling and verification of perception-based autonomous systems. We assume the perception model can be decomposed into a canonical model (obtained from first principles or a simulator) and a noise model that contains the measurement noise introduced by the real environment. We focus on two types of noise, benign and adversarial noise, and develop a data-driven model for each type using generative models and classifiers, respectively. We show that the trained models perform well according to a variety of evaluation metrics based on downstream tasks such as state estimation and control. Finally, we verify the safety of two systems with high-dimensional data-driven models, namely an image-based version of mountain car (a reinforcement learning benchmark) as well as the F1/10 car, which uses LiDAR measurements to navigate a racing track.
Accurate estimation of thermospheric density is critical for precise modeling of satellite drag forces in low Earth orbit (LEO). Improving this estimation is crucial to tasks such as state estimation, collision avoidance, and re-entry calculations. The largest source of uncertainty in determining thermospheric density is modeling the effects of space weather driven by solar and geomagnetic activity. Current operational models rely on ground-based proxy indices which imperfectly correlate with the complexity of solar outputs and geomagnetic responses. In this work, we directly incorporate NASA's Solar Dynamics Observatory (SDO) extreme ultraviolet (EUV) spectral images into a neural thermospheric density model to determine whether the predictive performance of the model is increased by using space-based EUV imagery data instead of, or in addition to, the ground-based proxy indices. We demonstrate that EUV imagery can enable predictions with much higher temporal resolution and replace ground-based proxies while significantly increasing performance relative to current operational models. Our method paves the way for assimilating EUV image data into operational thermospheric density forecasting models for use in LEO satellite navigation processes.
The typical point cloud sampling methods used in state estimation for mobile robots preserve a high level of point redundancy. This redundancy unnecessarily slows down the estimation pipeline and may cause drift under real-time constraints. Such undue latency becomes a bottleneck for resource-constrained robots (especially UAVs), requiring minimal delay for agile and accurate operation. We propose a novel, deterministic, uninformed, and single-parameter point cloud sampling method named RMS that minimizes redundancy within a 3D point cloud. In contrast to the state of the art, RMS balances the translation-space observability by leveraging the fact that linear and planar surfaces inherently exhibit high redundancy propagated into iterative estimation pipelines. We define the concept of gradient flow, quantifying the local surface underlying a point. We also show that maximizing the entropy of the gradient flow minimizes point redundancy for robot ego-motion estimation. We integrate RMS into the point-based KISS-ICP and feature-based LOAM odometry pipelines and evaluate experimentally on KITTI, Hilti-Oxford, and custom datasets from multirotor UAVs. The experiments demonstrate that RMS outperforms state-of-the-art methods in speed, compression, and accuracy in well-conditioned as well as in geometrically-degenerated settings.
State estimation has a pivotal role in several applications, including but not limited to advanced control design. Especially when dealing with nonlinear systems state estimation is a nontrivial task, often entailing approximations and challenging fine-tuning phases. In this work, we propose to overcome these challenges by formulating an in-context state-estimation problem, enabling us to learn a state estimator for a class of (nonlinear) systems abstracting from particular instances of the state seen during training. To this end, we extend an in-context learning framework recently proposed for system identification, showing via a benchmark numerical example that this approach allows us to (i) use training data directly for the design of the state estimator, (ii) not requiring extensive fine-tuning procedures, while (iii) achieving superior performance compared to state-of-the-art benchmarks.
Optimal decision-making presents a significant challenge for autonomous systems operating in uncertain, stochastic and time-varying environments. Environmental variability over time can significantly impact the system's optimal decision making strategy for mission completion. To model such environments, our work combines the previous notion of Time-Varying Markov Decision Processes (TVMDP) with partial observability and introduces Time-Varying Partially Observable Markov Decision Processes (TV-POMDP). We propose a two-pronged approach to accurately estimate and plan within the TV-POMDP: 1) Memory Prioritized State Estimation (MPSE), which leverages weighted memory to provide more accurate time-varying transition estimates; and 2) an MPSE-integrated planning strategy that optimizes long-term rewards while accounting for temporal constraint. We validate the proposed framework and algorithms using simulations and hardware, with robots exploring a partially observable, time-varying environments. Our results demonstrate superior performance over standard methods, highlighting the framework's effectiveness in stochastic, uncertain, time-varying domains.
This work studies the modeling of false data injection attacks (FDIAs) in integrated electricity-gas systems (IEGSs). First, we introduce a static state estimation model and bad data detection method for IEGSs. Then, we develop FDIAs on IEGSs with complete network topology and parameter information. Next, we develop FDIAs on IEGSs when intruders have only local network topology and parameter information of an IEGS. Lastly, we explore FDIAs on IEGSs when intruders have only local network topology information of an IEGS.
This paper investigates the state estimation problem for unknown linear systems subject to both process and measurement noise. Based on a prior input-output trajectory sampled at a higher frequency and a prior state trajectory sampled at a lower frequency, we propose a novel robust data-driven Kalman filter (RDKF) that integrates model identification with state estimation for the unknown system. Specifically, the state estimation problem is formulated as a non-convex maximum likelihood optimization problem. Then, we slightly modify the optimization problem to get a problem solvable with a recursive algorithm. Based on the optimal solution to this new problem, the RDKF is designed, which can estimate the state of a given but unknown state-space model. The performance gap between the RDKF and the optimal Kalman filter based on known system matrices is quantified through a sample complexity bound. In particular, when the number of the pre-collected states tends to infinity, this gap converges to zero. Finally, the effectiveness of the theoretical results is illustrated by numerical simulations.
Many mainstream robust control/estimation algorithms for power networks are designed using the Lyapunov theory as it provides performance guarantees for linear/nonlinear models of uncertain power networks but comes at the expense of scalability and sensitivity. In particular, Lyapunov-based approaches rely on forming semi-definite programs (SDPs) that are (i) not scalable and (ii) extremely sensitive to the choice of the bounding scalar that ensures the strict feasibility of the linear matrix inequalities (LMIs). This paper addresses these two issues by employing a celebrated non-Lyapunov approach (NLA) from the control theory literature. In lieu of linearized models of power grids, we focus on (the more representative) nonlinear differential algebraic equation (DAE) models and showcase the simplicity, scalability, and parameter-resiliency of NLA. For some power systems, the approach is nearly fifty times faster than solving SDPs via standard solvers with almost no impact on the performance. The case studies also demonstrate that NLA can be applied to more realistic scenarios in which (i) only partial state data is available and (ii) sparsity structures are imposed on the feedback gain. The paper also showcases that virtually no degradation in state estimation quality is experienced when applying NLA.
Knowledge of remaining battery charge is fundamental to electric vehicle deployment. Accurate measurements of state-of-charge (SOC) cannot be directly obtained, and estimation methods must be used instead. This requires both a good model of a battery and a well-designed state estimator. Here, hysteretic reduced-order battery models and adaptive extended Kalman filter estimators are shown to be highly effective, accurate SOC estimators. A battery model parameterisation framework is proposed, which enhances standardised methods to capture hysteresis effects. The hysteretic model is parameterised for three independent NMC811 lithium-ion cells and is shown to reduce voltage RMS error by 50% across 18-hour automotive drive-cycles. Parameterised models are used alongside an extended Kalman filter, which demonstrates the value of adaptive filter parameterisation schemes. When used alongside an extended Kalman filter, adaptive covariance matrices yield highly accurate SOC estimates, reducing SOC estimation error by 85%, compared to the industry standard battery model.
State inference and parameter learning in sequential models can be successfully performed with approximation techniques that maximize the evidence lower bound to the marginal log-likelihood of the data distribution. These methods may be referred to as Dynamical Variational Autoencoders, and our specific focus lies on the deep Kalman filter. It has been shown that the ELBO objective can oversimplify data representations, potentially compromising estimation quality. Tighter Monte Carlo objectives have been proposed in the literature to enhance generative modeling performance. For instance, the IWAE objective uses importance weights to reduce the variance of marginal log-likelihood estimates. In this paper, importance sampling is applied to the DKF framework for learning deep Markov models, resulting in the IW-DKF, which shows an improvement in terms of log-likelihood estimates and KL divergence between the variational distribution and the transition model. The framework using the sampled DKF update rule is also accommodated to address sequential state and parameter estimation when working with highly non-linear physics-based models. An experiment with the 3-space Lorenz attractor shows an enhanced generative modeling performance and also a decrease in RMSE when estimating the model parameters and latent states, indicating that tighter MCOs lead to improved state inference performance.
The setup considered in the paper consists of sensors in a Networked Control System that are used to build a digital twin (DT) model of the system dynamics. The focus is on control, scheduling, and resource allocation for sensory observation to ensure timely delivery to the DT model deployed in the cloud. Low latency and communication timeliness are instrumental in ensuring that the DT model can accurately estimate and predict system states. However, acquiring data for efficient state estimation and control computing poses a non-trivial problem given the limited network resources, partial state vector information, and measurement errors encountered at distributed sensors. We propose the REinforcement learning and Variational Extended Kalman filter with Robust Belief (REVERB), which leverages a reinforcement learning solution combined with a Value of Information-based algorithm for performing optimal control and selecting the most informative sensors to satisfy the prediction accuracy of DT. Numerical results demonstrate that the DT platform can offer satisfactory performance while reducing the communication overhead up to five times.
In this paper, we present a nodal hydraulic head estimation methodology for water distribution networks (WDN) based on an Unscented Kalman Filter (UKF) scheme with application to leak localization. The UKF refines an initial estimation of the hydraulic state by considering the prediction model, as well as available pressure and demand measurements. To this end, it provides customized prediction and data assimilation steps. Additionally, the method is enhanced by dynamically updating the prediction function weight matrices. Performance testing on the Modena benchmark under realistic conditions demonstrates the method's effectiveness in enhancing state estimation and data-driven leak localization.
This paper considers the problem of optimal sensor schedules for remote state estimation of discrete-event systems. In this setting, the sensors observe information from the plant and transmit the observable information to the receiver or estimator selectively. A transmission mechanism decides whether the observable information is transmitted or not, according to an information transmission policy, such that the receiver has sufficient information to satisfy the purpose of decision-making. To construct such a transmission mechanism, we first construct a non-deterministic dynamic observer that contains all feasible information transmission policies. Then, we show that the information updating rule of the dynamic observer indeed yields the state estimate from the receiver's point of view. Finally, we propose an approach to extract a specific information transmission policy, realized by a finite-state automaton, from the dynamic observer while satisfying some desired observation properties. To reduce transmission-related costs, we also require that the sensors transmit events as few as possible. A running example is provided to illustrate the proposed procedures.
The increasing penetration of renewable energy resources in distribution systems necessitates high-speed monitoring and control of voltage for ensuring reliable system operation. However, existing voltage control algorithms often make simplifying assumptions in their formulation, such as real-time availability of smart meter measurements (for monitoring), or real-time knowledge of every power injection information(for control).This paper leverages the recent advances made in highspeed state estimation for real-time unobservable distribution systems to formulate a deep reinforcement learning-based control algorithm that utilizes the state estimates alone to control the voltage of the entire system. The results obtained for a modified (renewable-rich) IEEE34-nodedistributionfeeder indicate that the proposed approach excels in monitoring and controlling voltage of active distribution systems.
Traditional smart meters, which measure energy usage every 15 minutes or more and report it at least a few hours later, lack the granularity needed for real-time decision-making. To address this practical problem, we introduce a new method using generative adversarial networks (GAN) that enforces temporal consistency on its high-resolution outputs via hard inequality constraints using convex optimization. A unique feature of our GAN model is that it is trained solely on slow timescale aggregated historical energy data obtained from smart meters. The results demonstrate that the model can successfully create minute-by-minute temporally correlated profiles of power usage from 15-minute interval average power consumption information. This innovative approach, emphasizing inter-neuron constraints, offers a promising avenue for improved high-speed state estimation in distribution systems and enhances the applicability of data-driven solutions for monitoring and subsequently controlling such systems.
Reconfigurable intelligent surfaces (RISs) are considered as an enabling technology for the upcoming sixth generation of wireless systems, exhibiting significant potential for radio localization and sensing. An RIS is usually treated as an anchor point with known position and orientation when deployed to offer user localization. However, it can also be attached to a user to enable its localization in a semi-passive manner. In this paper, we consider a static user equipped with an RIS and study the RIS localization problem (i.e., joint three-dimensional position and orientation estimation), when operating in a system comprising a single-antenna transmitter and multiple synchronized single-antenna receivers with known locations. We present a multi-stage estimator using time-of-arrival and spatial frequency measurements, and derive the Cram\'er-Rao lower bounds for the estimated parameters to validate the estimator's performance. Our simulation results demonstrate the efficiency of the proposed RIS state estimation approach under various system operation parameters.
We extend the notion of estimation entropy of autonomous dynamical systems proposed by Liberzon and Mitra [1] to nonlinear dynamical systems with uncertain inputs with bounded variation. We call this new notion the {$\epsilon$}-estimation entropy of the system and show that it lower bounds the bit rate needed for state estimation. {$\epsilon$}-estimation entropy represents the exponential rate of the increase of the minimal number of functions that are adequate for {$\epsilon$}- approximating any trajectory of the system. We show that alternative entropy definitions using spanning or separating trajectories bound ours from both sides. On the other hand, we show that other commonly used definitions of entropy, for example the ones in [1], diverge to infinity. Thus, they are potentially not suitable for systems with uncertain inputs. We derive an upper bound on {$\epsilon$}-estimation entropy and estimation bit rates, and evaluate it for two examples. We present a state estimation algorithm that constructs a function that approximates a given trajectory up to an {$\epsilon$} error, given time-sampled and quantized measurements of state and input. We investigate the relation between {$\epsilon$}-estimation entropy and a previous notion for switched nonlinear systems and derive a new upper bound for the latter, showing the generality of our results on systems with uncertain inputs.