Magnetic Resonance Fingerprinting (MRF) is a novel technique that simultaneously estimates multiple tissue-related parameters, such as the longitudinal relaxation time T1, the transverse relaxation time T2, off resonance frequency B0 and proton density, from a scanned object in just tens of seconds. However, the MRF method suffers from aliasing artifacts because it significantly undersamples the k-space data. In this work, we propose a compressed sensing (CS) framework for simultaneously estimating multiple tissue-related parameters based on the MRF method. It is more robust to low sampling ratio and is therefore more efficient in estimating MR parameters for all voxels of an object. Furthermore, the MRF method requires identifying the nearest atoms of the query fingerprints from the MR-signal-evolution dictionary with the L2 distance. However, we observed that the L2 distance is not always a proper metric to measure the similarities between MR Fingerprints. Adaptively learning a distance metric from the undersampled training data can significantly improve the matching accuracy of the query fingerprints. Numerical results on extensive simulated cases show that our method substantially outperforms stateof-the-art methods in terms of accuracy of parameter estimation.
We investigate a novel non-parametric regression-based clustering algorithm for longitudinal data analysis. Combining natural cubic splines with Gaussian mixture models (GMM), the algorithm can produce smooth cluster means that describe the underlying data well. However, there are some shortcomings in the algorithm: high computational complexity in the parameter estimation procedure and a numerically unstable variance estimator. Therefore, to further increase the usability of the method, we incorporated approaches to reduce its computational complexity, we developed a new, more stable variance estimator, and we developed a new smoothing parameter estimation procedure. We show that the developed algorithm, SMIXS, performs better than GMM on a synthetic dataset in terms of clustering and regression performance. We demonstrate the impact of the computational speed-ups, which we formally prove in the new framework. Finally, we perform a case study by using SMIXS to cluster vertical atmospheric measurements to determine different weather regimes.
This paper studies the estimation of cascaded channels in passive intelligent reflective surface (IRS)- aided multiple-input multiple-output (MIMO) systems employing hybrid precoders and combiners. We propose a low-complexity solution that estimates the channel parameters progressively. The angles of departure (AoDs) and angles of arrival (AoAs) at the transmitter and receiver, respectively, are first estimated using inductive matrix completion (IMC) followed by root-MUSIC based super-resolution spectrum estimation. Forward-backward spatial smoothing (FBSS) is applied to address the coherence issue. Using the estimated AoAs and AoDs, the training precoders and combiners are then optimized and the angle differences between the AoAs and AoDs at the IRS are estimated using the least squares (LS) method followed by FBSS and the root-MUSIC algorithm. Finally, the composite path gains of the cascaded channel are estimated using on-grid sparse recovery with a small-size dictionary. The simulation results suggest that the proposed estimator can achieve improved channel parameter estimation performance with lower complexity as compared to several recently reported alternatives, thanks to the exploitation of the knowledge of the array responses and low-rankness of the channel using low-complexity algorithms at all the stages.
This paper deals with the problem of on-line identification of the parameters of a realistic dynamical model of a photovoltaic array connected to a power system through a power converter. It has been shown in the literature that, when interacting with switching devices, this model is able to better account for the PV array operation, as compared to the classical five parameter static model of the array. While there are many results of identification of the parameters of the latter model, to the best of our knowledge, no one has provided a solution for the aforementioned more complex dynamic model since it concerns the parameter estimation of a nonlinear, underexcited system with unmeasurable state variables. Achieving such objective is the main contribution of the paper. We propose a new parameterisation of the dynamic model, which, combined with the powerful identification technique of dynamic regressor extension and mixing, ensures a fast and accurate online estimation of the unknown parameters. Realistic numerical examples via computer simulations are presented to assess the performance of the proposed approach -- even being able to track the parameter variations when the system changes operating point.
Optical coherence tomography (OCT) is a micrometer-scale, volumetric imaging modality that has become a clinical standard in ophthalmology. OCT instruments image by raster-scanning a focused light spot across the retina, acquiring sequential cross-sectional images to generate volumetric data. Patient eye motion during the acquisition poses unique challenges: Non-rigid, discontinuous distortions can occur, leading to gaps in data and distorted topographic measurements. We present a new distortion model and a corresponding fully-automatic, reference-free optimization strategy for computational motion correction in orthogonally raster-scanned, retinal OCT volumes. Using a novel, domain-specific spatiotemporal parametrization of forward-warping displacements, eye motion can be corrected continuously for the first time. Parameter estimation with temporal regularization improves robustness and accuracy over previous spatial approaches. We correct each A-scan individually in 3D in a single mapping, including repeated acquisitions used in OCT angiography protocols. Specialized 3D forward image warping reduces median runtime to < 9 s, fast enough for clinical use. We present a quantitative evaluation on 18 subjects with ocular pathology and demonstrate accurate correction during microsaccades. Transverse correction is limited only by ocular tremor, whereas submicron repeatability is achieved axially (0.51 um median of medians), representing a dramatic improvement over previous work. This allows assessing longitudinal changes in focal retinal pathologies as a marker of disease progression or treatment response, and promises to enable multiple new capabilities such as supersampled/super-resolution volume reconstruction and analysis of pathological eye motion occuring in neurological diseases.
We study the problem of estimating an unknown parameter in a distributed and online manner. Existing work on distributed online learning typically either focuses on asymptotic analysis, or provides bounds on regret. However, these results may not directly translate into bounds on the error of the learned model after a finite number of time-steps. In this paper, we propose a distributed online estimation algorithm which enables each agent in a network to improve its estimation accuracy by communicating with neighbors. We provide non-asymptotic bounds on the estimation error, leveraging the statistical properties of the underlying model. Our analysis demonstrates a trade-off between estimation error and communication costs. Further, our analysis allows us to determine a time at which the communication can be stopped (due to the costs associated with communications), while meeting a desired estimation accuracy. We also provide a numerical example to validate our results.
Given a non-contracting trajectory of a nonlinear system, we consider the question of designing an input perturbation that makes the perturbed trajectory contracting. This paper stresses the analogy of this question with the classical question of feedback stabilization. In particular, it is shown that the existence of an output variable that ensures contraction of the inverse system facilitates the design of a contracting input perturbation. We illustrate the relevance of this question in parameter estimation.
One of the biggest challenges in characterizing 2-D topographies is succinctly communicating the dominant nature of local configurations. In a 2-D grid composed of bistate units, this could be expressed as finding the characteristic configuration variables such as nearest-neighbor pairs and triplet combinations. The 2-D cluster variation method (CVM) provides a theoretical framework for associating a set of configuration variables with only two parameters, for a system that is at free energy equilibrium. This work presents a method for determining which of many possible two-parameter sets provides the ``most suitable'' match for a given 2-D topography, drawing from methods used for variational inference. This particular work focuses exclusively on topographies for which the activation enthalpy parameter (epsilon_0) is zero, so that the distribution between two states is equiprobable. This condition is used since, when the two states are equiprobable, there is an analytic solution giving the configuration variable values as functions of the h-value, where we define h in terms of the interaction enthalpy parameter (epsilon_1) as h = exp(2*epsilon_1). This allows the computationally-achieved configuration variable values to be compared with the analytically-predicted values for a given h-value. The method is illustrated using four patterns derived from three different naturally-occurring black-and-white topographies, where each pattern meets the equiprobability criterion. We achieve expected results, that is, as the patterns progress from having relatively low numbers of like-near-like nodes to increasing like-near-like masses, the h-values for each corresponding free energy-minimized model also increase. Further, the corresponding configuration variable values for the (free energy-minimized) model patterns are in approximate alignment with the analytically-predicted values.
The application of the context-adaptive entropy model significantly improves the rate-distortion (R-D) performance, in which hyperpriors and autoregressive models are jointly utilized to effectively capture the spatial redundancy of the latent representations. However, the latent representations still contain some spatial correlations. In addition, these methods based on the context-adaptive entropy model cannot be accelerated in the decoding process by parallel computing devices, e.g. FPGA or GPU. To alleviate these limitations, we propose a learned multi-resolution image compression framework, which exploits the recently developed octave convolutions to factorize the latent representations into the high-resolution (HR) and low-resolution (LR) parts, similar to wavelet transform, which further improves the R-D performance. To speed up the decoding, our scheme does not use context-adaptive entropy model. Instead, we exploit an additional hyper layer including hyper encoder and hyper decoder to further remove the spatial redundancy of the latent representation. Moreover, the cross-resolution parameter estimation (CRPE) is introduced into the proposed framework to enhance the flow of information and further improve the rate-distortion performance. An additional information-fidelity loss is proposed to the total loss function to adjust the contribution of the LR part to the final bit stream. Experimental results show that our method separately reduces the decoding time by approximately 73.35 % and 93.44 % compared with that of state-of-the-art learned image compression methods, and the R-D performance is still better than H.266/VVC(4:2:0) and some learning-based methods on both PSNR and MS-SSIM metrics across a wide bit rates.
In many scenarios, the communication system suffers from both Gaussian white noise and non-Gaussian impulsive noise. In order to design optimal signal detection method, it is necessary to estimate the parameters of mixed Gaussian-impulsive noise. Even though this issue can be well tackled with respect to pure mixed noise, it is quite challenging based on the received single-channel signal including both transmitting signal and mixed noise. To mitigate the negative impact of transmitting signal, we propose a parameter estimation method by utilizing a neural network, namely U-net++, to separate the mixed noise from the received single-channel signal. Compared with existing blind source separation based methods, simulation results show that our proposed method can obtain rather better performance in terms of estimation accuracy and robustness under various scenarios.
The Expectation--Maximization (EM) algorithm is a simple meta-algorithm that has been used for many years as a methodology for statistical inference when there are missing measurements in the observed data or when the data is composed of observables and unobservables. Its general properties are well studied, and also, there are countless ways to apply it to individual problems. In this paper, we introduce the $em$ algorithm, an information geometric formulation of the EM algorithm, and its extensions and applications to various problems. Specifically, we will see that it is possible to formulate an outlier-robust inference algorithm, an algorithm for calculating channel capacity, parameter estimation methods on probability simplex, particular multivariate analysis methods such as principal component analysis in a space of probability models and modal regression, matrix factorization, and learning generative models, which have recently attracted attention in deep learning, from the geometric perspective.
Dynamic state and parameter estimation methods for dynamic security assessment in power systems are becoming increasingly important for system operators. Usually, the data used for this type of applications stems from phasor measurement units (PMUs) and is corrupted by noise. In general, the impact of the latter may significantly deteriorate the estimation performance. This motivates the present work, in which it is proven that the state and parameter estimation method proposed by part of the authors in [1] and extended in [2] features the property that the estimation errors are ultimately bounded in the presence of PMU measurement data corrupted by bounded noise. The analysis is conducted for the third-order flux-decay model of a synchronous generator and holds independently of the employed automatic voltage regulator and power system stabilizer (if present). The analysis is illustrated by simulations.
This article presents XGB-Chiarella, a powerful new approach for deploying agent-based models to generate realistic intra-day artificial financial price data. This approach is based on agent-based models, calibrated by XGBoost machine learning surrogate. Following the Extended Chiarella model, three types of trading agents are introduced in this agent-based model: fundamental traders, momentum traders, and noise traders. In particular, XGB-Chiarella focuses on configuring the simulation to accurately reflect real market behaviours. Instead of using the original Expectation-Maximisation algorithm for parameter estimation, the agent-based Extended Chiarella model is calibrated using XGBoost machine learning surrogate. It is shown that the machine learning surrogate learned in the proposed method is an accurate proxy of the true agent-based market simulation. The proposed calibration method is superior to the original Expectation-Maximisation parameter estimation in terms of the distance between historical and simulated stylised facts. With the same underlying model, the proposed methodology is capable of generating realistic price time series in various stocks listed at three different exchanges, which indicates the universality of intra-day price formation process. For the time scale (minutes) chosen in this paper, one agent per category is shown to be sufficient to capture the intra-day price formation process. The proposed XGB-Chiarella approach provides insights that the price formation process is comprised of the interactions between momentum traders, fundamental traders, and noise traders. It can also be used to enhance risk management by practitioners.
This paper considers an affine frequency division multiplexing (AFDM)-based integrated sensing and communications (ISAC) system, where the AFDM waveform is used to simultaneously carry communications information and sense targets. To realize AFDM-based sensing functionality, two parameter estimation methods are designed to process echoes in the time domain and the discrete affine Fourier transform (DAFT) domain, respectively. It allows us to decouple delay and Doppler shift in the fast time axis and can maintain good sensing performance even in large Doppler shift scenarios. Numerical results verify the effectiveness of our proposed AFDM-based system in high mobility scenarios.
Accurate knowledge of transmission line parameters is essential for a variety of power system monitoring, protection, and control applications. The use of phasor measurement unit (PMU) data for transmission line parameter estimation (TLPE) is well-documented. However, existing literature on PMU-based TLPE implicitly assumes the measurement noise to be Gaussian. Recently, it has been shown that the noise in PMU measurements (especially in the current phasors) is better represented by Gaussian mixture models (GMMs), i.e., the noises are non-Gaussian. We present a novel approach for TLPE that can handle non-Gaussian noise in the PMU measurements. The measurement noise is expressed as a GMM, whose components are identified using the expectation-maximization (EM) algorithm. Subsequently, noise and parameter estimation is carried out by solving a maximum likelihood estimation problem iteratively until convergence. The superior performance of the proposed approach over traditional approaches such as least squares and total least squares as well as the more recently proposed minimum total error entropy approach is demonstrated by performing simulations using the IEEE 118-bus system as well as proprietary PMU data obtained from a U.S. power utility.
Neural point estimators are neural networks that map data to parameter point estimates. They are fast, likelihood free and, due to their amortised nature, amenable to fast bootstrap-based uncertainty quantification. In this paper, we aim to increase the awareness of statisticians to this relatively new inferential tool, and to facilitate its adoption by providing user-friendly open-source software. We also give attention to the ubiquitous problem of making inference from replicated data, which we address in the neural setting using permutation-invariant neural networks. Through extensive simulation studies we show that these neural point estimators can quickly and optimally (in a Bayes sense) estimate parameters in weakly-identified and highly-parameterised models with relative ease. We demonstrate their applicability through an analysis of extreme sea-surface temperature in the Red Sea where, after training, we obtain parameter estimates and bootstrap-based confidence intervals from hundreds of spatial fields in a fraction of a second.
Structural monitoring for complex built environments often suffers from mismatch between design, laboratory testing, and actual built parameters. Additionally, real-world structural identification problems encounter many challenges. For example, the lack of accurate baseline models, high dimensionality, and complex multivariate partial differential equations (PDEs) pose significant difficulties in training and learning conventional data-driven algorithms. This paper explores a new framework, dubbed NeuralSI, for structural identification by augmenting PDEs that govern structural dynamics with neural networks. Our approach seeks to estimate nonlinear parameters from governing equations. We consider the vibration of nonlinear beams with two unknown parameters, one that represents geometric and material variations, and another that captures energy losses in the system mainly through damping. The data for parameter estimation is obtained from a limited set of measurements, which is conducive to applications in structural health monitoring where the exact state of an existing structure is typically unknown and only a limited amount of data samples can be collected in the field. The trained model can also be extrapolated under both standard and extreme conditions using the identified structural parameters. We compare with pure data-driven Neural Networks and other classical Physics-Informed Neural Networks (PINNs). Our approach reduces both interpolation and extrapolation errors in displacement distribution by two to five orders of magnitude over the baselines. Code is available at https://github.com/human-analysis/neural-structural-identification
In this paper, we consider the task of clustering a set of individual time series while modeling each cluster, that is, model-based time series clustering. The task requires a parametric model with sufficient flexibility to describe the dynamics in various time series. To address this problem, we propose a novel model-based time series clustering method with mixtures of linear Gaussian state space models, which have high flexibility. The proposed method uses a new expectation-maximization algorithm for the mixture model to estimate the model parameters, and determines the number of clusters using the Bayesian information criterion. Experiments on a simulated dataset demonstrate the effectiveness of the method in clustering, parameter estimation, and model selection. The method is applied to real datasets commonly used to evaluate time series clustering methods. Results showed that the proposed method produces clustering results that are as accurate or more accurate than those obtained using previous methods.
Discovering the underlying dynamics of complex systems from data is an important practical topic. Constrained optimization algorithms are widely utilized and lead to many successes. Yet, such purely data-driven methods may bring about incorrect physics in the presence of random noise and cannot easily handle the situation with incomplete data. In this paper, a new iterative learning algorithm for complex turbulent systems with partial observations is developed that alternates between identifying model structures, recovering unobserved variables, and estimating parameters. First, a causality-based learning approach is utilized for the sparse identification of model structures, which takes into account certain physics knowledge that is pre-learned from data. It has unique advantages in coping with indirect coupling between features and is robust to the stochastic noise. A practical algorithm is designed to facilitate the causal inference for high-dimensional systems. Next, a systematic nonlinear stochastic parameterization is built to characterize the time evolution of the unobserved variables. Closed analytic formula via an efficient nonlinear data assimilation is exploited to sample the trajectories of the unobserved variables, which are then treated as synthetic observations to advance a rapid parameter estimation. Furthermore, the localization of the state variable dependence and the physics constraints are incorporated into the learning procedure, which mitigate the curse of dimensionality and prevent the finite time blow-up issue. Numerical experiments show that the new algorithm succeeds in identifying the model structure and providing suitable stochastic parameterizations for many complex nonlinear systems with chaotic dynamics, spatiotemporal multiscale structures, intermittency, and extreme events.
We investigate the inverse problem for Partial Differential Equations (PDEs) in scenarios where the parameters of the given PDE dynamics may exhibit changepoints at random time. We employ Physics-Informed Neural Networks (PINNs) - universal approximators capable of estimating the solution of any physical law described by a system of PDEs, which serves as a regularization during neural network training, restricting the space of admissible solutions and enhancing function approximation accuracy. We demonstrate that when the system exhibits sudden changes in the PDE dynamics, this regularization is either insufficient to accurately estimate the true dynamics, or it may result in model miscalibration and failure. Consequently, we propose a PINNs extension using a Total-Variation penalty, which allows to accommodate multiple changepoints in the PDE dynamics and significantly improves function approximation. These changepoints can occur at random locations over time and are estimated concurrently with the solutions. Additionally, we introduce an online learning method for re-weighting loss function terms dynamically. Through empirical analysis using examples of various equations with parameter changes, we showcase the advantages of our proposed model. In the absence of changepoints, the model reverts to the original PINNs model. However, when changepoints are present, our approach yields superior parameter estimation, improved model fitting, and reduced training error compared to the original PINNs model.