Multi-function radars (MFRs) are sophisticated types of sensors with the capabilities of complex agile inter-pulse modulation implementation and dynamic work mode scheduling. The developments in MFRs pose great challenges to modern electronic reconnaissance systems or radar warning receivers for recognition and inference of MFR work modes. To address this issue, this paper proposes an online processing framework for parameter estimation and change point detection of MFR work modes. At first, this paper designed a fully-conjugate Bayesian non-parametric hidden Markov model with a designed prior distribution (agile BNP-HMM) to represent the MFR pulse agility characteristics. The proposed model allows fully-variational Bayesian inference. Then, the proposed framework is constructed by two main parts. The first part is the agile BNP-HMM model for automatically inferring the number of HMM hidden states and emission distribution of the corresponding hidden states. An estimation error lower bound on performance is derived and the proposed algorithm is shown to be close to the bound. The second part utilizes the streaming Bayesian updating to facilitate computation, and designed an online work mode change detection framework based upon a weighted sequential probability ratio test. We demonstrate that the proposed framework is consistently highly effective and robust to baseline methods on diverse simulated data-sets.
In this paper, we introduce a method to optimally estimate time-varying frequency bias. Current industry practice is to assume that frequency bias is changing only on annual basis. We suggest that this improved time-dependent bias estimate can be used to reduce the cost of frequency regulation needed to meet industry standards requested by the North American Electric Reliability Corporation (NERC). Optimization of time-varying frequency bias is posed as a parameter estimation (calibration) problem whose implementation utilizes online system measurements. It is further shown how this result can be used to estimate intra-dispatch load deviations. This knowledge is needed to estimate more accurately regulation reserve needed, and to therefore reduce overall regulation cost. Methods can be introduced to give incentives to demand response to participate in frequency regulation. Overall, we show the importance of incorporating knowledge of physics-based models for data-enabled parameter estimation of physical systems.
Simulation-based inference techniques are indispensable for parameter estimation of mechanistic and simulable models with intractable likelihoods. While traditional statistical approaches like approximate Bayesian computation and Bayesian synthetic likelihood have been studied under well-specified and misspecified settings, they often suffer from inefficiencies due to wasted model simulations. Neural approaches, such as sequential neural likelihood (SNL) avoid this wastage by utilising all model simulations to train a neural surrogate for the likelihood function. However, the performance of SNL under model misspecification is unreliable and can result in overconfident posteriors centred around an inaccurate parameter estimate. In this paper, we propose a novel SNL method, which through the incorporation of additional adjustment parameters, is robust to model misspecification and capable of identifying features of the data that the model is not able to recover. We demonstrate the efficacy of our approach through several illustrative examples, where our method gives more accurate point estimates and uncertainty quantification than SNL.
Probabilistic graphical models (PGMs) provide a compact and flexible framework to model very complex real-life phenomena. They combine the probability theory which deals with uncertainty and logical structure represented by a graph which allows one to cope with the computational complexity and also interpret and communicate the obtained knowledge. In the thesis, we consider two different types of PGMs: Bayesian networks (BNs) which are static, and continuous time Bayesian networks which, as the name suggests, have a temporal component. We are interested in recovering their true structure, which is the first step in learning any PGM. This is a challenging task, which is interesting in itself from the causal point of view, for the purposes of interpretation of the model and the decision-making process. All approaches for structure learning in the thesis are united by the same idea of maximum likelihood estimation with the LASSO penalty. The problem of structure learning is reduced to the problem of finding non-zero coefficients in the LASSO estimator for a generalized linear model. In the case of CTBNs, we consider the problem both for complete and incomplete data. We support the theoretical results with experiments.
This work proposes a switched model reference adaptive control (S-MRAC) architecture for a multi-input multi-output (MIMO) switched linear system with memory for enhanced learning. A salient feature of the proposed method that separates it from most previous results is the use of memory that store the estimator states at switching and facilitate parameter learning during both active and inactive phases of a subsystem, thereby improving the tracking performance of the overall switched system. Specifically, the learning experience from the previous active duration of a subsystem is retained in the memory and reused when the subsystem is inactive and when the subsystem becomes active again. Parameter convergence is shown based on an intermittent initial excitation (IIE), which is significantly relaxed than the classical persistence of excitation (PE) condition. A common Lyapunov function is considered to ensure closed-loop stability with S-MRAC. Further under IIE, the exponential stability of tracking and parameter estimation error dynamics are guaranteed.
Likelihood-free inference methods typically make use of a distance between simulated and real data. A common example is the maximum mean discrepancy (MMD), which has previously been used for approximate Bayesian computation, minimum distance estimation, generalised Bayesian inference, and within the nonparametric learning framework. The MMD is commonly estimated at a root-$m$ rate, where $m$ is the number of simulated samples. This can lead to significant computational challenges since a large $m$ is required to obtain an accurate estimate, which is crucial for parameter estimation. In this paper, we propose a novel estimator for the MMD with significantly improved sample complexity. The estimator is particularly well suited for computationally expensive smooth simulators with low- to mid-dimensional inputs. This claim is supported through both theoretical results and an extensive simulation study on benchmark simulators.
In this paper, we investigate the integration of integrated sensing and communication (ISAC) and reconfigurable intelligent surfaces (RIS) for providing wide-coverage and ultra-reliable communication and high-accuracy sensing functions. In particular, we consider an RIS-assisted ISAC system in which a multi-antenna base station (BS) simultaneously performs multi-user multi-input single-output (MU-MISO) communications and radar sensing with the assistance of an RIS. We focus on both target detection and parameter estimation performance in terms of the signal-to-noise ratio (SNR) and Cramer-Rao bound (CRB), respectively. Two optimization problems are formulated for maximizing the achievable sum-rate of the multi-user communications under an SNR constraint for target detection or a CRB constraint for parameter estimation, the transmit power budget, and the unit-modulus constraint of the RIS reflection coefficients. Efficient algorithms are developed to solve these two complicated non-convex problems. Extensive simulation results demonstrate the advantages of the proposed joint beamforming and reflection designs compared with other schemes. In addition, it is shown that more RIS reflection elements bring larger performance gains for direct-of-arrival (DoA) estimation than for target detection.
We present a model inversion algorithm, CKLEMAP, for data assimilation and parameter estimation in partial differential equation models of physical systems with spatially heterogeneous parameter fields. These fields are approximated using low-dimensional conditional Karhunen-Lo\'{e}ve expansions, which are constructed using Gaussian process regression models of these fields trained on the parameters' measurements. We then assimilate measurements of the state of the system and compute the maximum a posteriori estimate of the CKLE coefficients by solving a nonlinear least-squares problem. When solving this optimization problem, we efficiently compute the Jacobian of the vector objective by exploiting the sparsity structure of the linear system of equations associated with the forward solution of the physics problem. The CKLEMAP method provides better scalability compared to the standard MAP method. In the MAP method, the number of unknowns to be estimated is equal to the number of elements in the numerical forward model. On the other hand, in CKLEMAP, the number of unknowns (CKLE coefficients) is controlled by the smoothness of the parameter field and the number of measurements, and is in general much smaller than the number of discretization nodes, which leads to a significant reduction of computational cost with respect to the standard MAP method. To show its advantage in scalability, we apply CKLEMAP to estimate the transmissivity field in a two-dimensional steady-state subsurface flow model of the Hanford Site by assimilating synthetic measurements of transmissivity and hydraulic head. We find that the execution time of CKLEMAP scales nearly linearly as $N^{1.33}$, where $N$ is the number of discretization nodes, while the execution time of standard MAP scales as $N^{2.91}$. The CKLEMAP method improved execution time without sacrificing accuracy when compared to the standard MAP.
We present a novel thermodynamic parameter estimation framework for energy-based surgery on live tissue, with direct applications to tissue characterization during electrosurgery. This framework addresses the problem of estimating tissue-specific thermodynamics in real-time, which would enable accurate prediction of thermal damage impact to the tissue and damage-conscious planning of electrosurgical procedures. Our approach provides basic thermodynamic information such as thermal diffusivity, and also allows for obtaining the thermal relaxation time and a model of the heat source, yielding in real-time a controlled hyperbolic thermodynamics model. The latter accounts for the finite thermal propagation time necessary for modeling of the electrosurgical action, in which the probe motion speed often surpasses the speed of thermal propagation in the tissue operated on. Our approach relies solely on thermographer feedback and a knowledge of the power level and position of the electrosurgical pencil, imposing only very minor adjustments to normal electrosurgery to obtain a high-fidelity model of the tissue-probe interaction. Our method is minimally invasive and can be performed in situ. We apply our method first to simulated data based on porcine muscle tissue to verify its accuracy and then to in vivo liver tissue, and compare the results with those from the literature. This comparison shows that parameterizing the Maxwell--Cattaneo model through the framework proposed yields a noticeably higher fidelity real-time adaptable representation of the thermodynamic tissue response to the electrosurgical impact than currently available. A discussion on the differences between the live and the dead tissue thermodynamics is also provided.
An adaptive state observer is proposed for a class of overparametrized uncertain linear time-invariant systems without restrictive requirement of their representation in the observer canonical form. It evolves the method of generalized parameters estimation-based observer design and, therefore, (i) does not require to identify Luenberger correction gain parameters, (ii) forms states using algebraic rather than differential equation. Additionally, the developed observer is applicable to systems with unknown output matrix and ensures exponential convergence of unmeasured state observation error under weak requirement of the regressor finite excitation. The effectiveness of the proposed solution is supported by simulation results.
The progressive integration of electronic converter based Renewable Energy Sources will lead to a reduction of the system synchronous inertia that can endanger the system frequency stability behavior, especially in the case of isolated or poorly interconnected systems, such as the case of Iberian Peninsula system. This work focuses on the study of the impact that the synchronous inertia reduction can have on the Iberian system frequency stability for different critical operation scenarios and disturbances. For this purpose, an equivalent dynamic model of the Iberian system interconnected with the central European network was first developed using the Matlab Simulink graphical programming environment. The parameters of this model are identified using a parameter estimation approach that explores the recorded behavior of the frequency following an outage of a nuclear unit in Spain. This model was used afterwards to perform simulations for different generation outages and multiple operation scenarios between 2020 and 2040.
We revisit conduct parameter estimation in homogeneous goods markets to resolve the conflict between Bresnahan (1982) and Perloff and Shen (2012) regarding the identification and the estimation of conduct parameters. We point out that Perloff and Shen's (2012) proof is incorrect and its simulation setting is invalid. Our simulation shows that estimation becomes accurate when demand shifters are properly added in supply estimation and sample sizes are increased, supporting Bresnahan (1982).
Robust M-estimation uses loss functions, such as least absolute deviation (LAD), quantile loss and Huber's loss, to construct its objective function, in order to for example eschew the impact of outliers, whereas the difficulty in analysing the resultant estimators rests on the nonsmoothness of these losses. Generalized functions have advantages over ordinary functions in several aspects, especially generalized functions possess derivatives of any order. Generalized functions incorporate local integrable functions, the so-called regular generalized functions, while the so-called singular generalized functions (e.g. Dirac delta function) can be obtained as the limits of a sequence of sufficient smooth functions, so-called regular sequence in generalized function context. This makes it possible to use these singular generalized functions through approximation. Nevertheless, a significant contribution of this paper is to establish the convergence rate of regular sequence to nonsmooth loss that answers a call from the relevant literature. For parameter estimation where objective function may be nonsmooth, this paper first shows as a general paradigm that how generalized function approach can be used to tackle the nonsmooth loss functions in Section two using a very simple model. This approach is of general interest and applicability. We further use the approach in robust M-estimation for additive single-index cointegrating time series models; the asymptotic theory is established for the proposed estimators. We evaluate the finite-sample performance of the proposed estimation method and theory by both simulated data and an empirical analysis of predictive regression of stock returns.
An individualized treatment regime (ITR) is a decision rule that assigns treatments based on patients' characteristics. The value function of an ITR is the expected outcome in a counterfactual world had this ITR been implemented. Recently, there has been increasing interest in combining heterogeneous data sources, such as leveraging the complementary features of randomized controlled trial (RCT) data and a large observational study (OS). Usually, a covariate shift exists between the source and target population, rendering the source-optimal ITR unnecessarily optimal for the target population. We present an efficient and robust transfer learning framework for estimating the optimal ITR with right-censored survival data that generalizes well to the target population. The value function accommodates a broad class of functionals of survival distributions, including survival probabilities and restrictive mean survival times (RMSTs). We propose a doubly robust estimator of the value function, and the optimal ITR is learned by maximizing the value function within a pre-specified class of ITRs. We establish the $N^{-1/3}$ rate of convergence for the estimated parameter indexing the optimal ITR, and show that the proposed optimal value estimator is consistent and asymptotically normal even with flexible machine learning methods for nuisance parameter estimation. We evaluate the empirical performance of the proposed method by simulation studies and a real data application of sodium bicarbonate therapy for patients with severe metabolic acidaemia in the intensive care unit (ICU), combining a RCT and an observational study with heterogeneity.
Dynamic operation of anaerobic digestion plants requires advanced process monitoring and control. Different simplifications of the Anaerobic Digestion Model No. 1 (ADM1) have been proposed recently, which appear promising for model-based process automation and state estimation. As a fundamental requirement, observability and identifiability of these models are analyzed in this work, which was pursued through algebraic and geometric analysis. Manual algebraic assessment was successfull for small models such as the ADM1-R4 and simplified versions of the ADM1-R3, which were derived in this context. However, for larger model classes the algebraic approach showed to be insufficient. By contrast, the geometric approach, implemented in the STRIKE_GOLDD toolbox, allowed to show observability for more complex models (including ADM1-R4 and ADM1-R3), employing two independent algorithms. The present study lays the groundwork for state observer design, parameter estimation and advanced control resting upon ADM1-based models.
Estimation of the parameters of a 2-dimensional sinusoidal model is a fundamental problem in digital signal processing and time series analysis. In this paper, we propose a robust least absolute deviation (LAD) estimators for parameter estimation. The proposed methodology provides a robust alternative to non-robust estimation techniques like the least squares estimators, in situations where outliers are present in the data or in the presence of heavy tailed noise. We study important asymptotic properties of the LAD estimators and establish the strong consistency and asymptotic normality of the LAD estimators of the signal parameters of a 2-dimensional sinusoidal model. We further illustrate the advantage of using LAD estimators over least squares estimators through extensive simulation studies. Data analysis of a 2-dimensional texture data indicates practical applicability of the proposed LAD approach.
The stochastic proximal point (SPP) methods have gained recent attention for stochastic optimization, with strong convergence guarantees and superior robustness to the classic stochastic gradient descent (SGD) methods showcased at little to no cost of computational overhead added. In this article, we study a minibatch variant of SPP, namely M-SPP, for solving convex composite risk minimization problems. The core contribution is a set of novel excess risk bounds of M-SPP derived through the lens of algorithmic stability theory. Particularly under smoothness and quadratic growth conditions, we show that M-SPP with minibatch-size $n$ and iteration count $T$ enjoys an in-expectation fast rate of convergence consisting of an $\mathcal{O}\left(\frac{1}{T^2}\right)$ bias decaying term and an $\mathcal{O}\left(\frac{1}{nT}\right)$ variance decaying term. In the small-$n$-large-$T$ setting, this result substantially improves the best known results of SPP-type approaches by revealing the impact of noise level of model on convergence rate. In the complementary small-$T$-large-$n$ regime, we provide a two-phase extension of M-SPP to achieve comparable convergence rates. Moreover, we derive a near-tight high probability (over the randomness of data) bound on the parameter estimation error of a sampling-without-replacement variant of M-SPP. Numerical evidences are provided to support our theoretical predictions when substantialized to Lasso and logistic regression models.
This paper considers the problem of system identification (ID) of linear and nonlinear non-autonomous systems from noisy and sparse data. We propose and analyze an objective function derived from a Bayesian formulation for learning a hidden Markov model with stochastic dynamics. We then analyze this objective function in the context of several state-of-the-art approaches for both linear and nonlinear system ID. In the former, we analyze least squares approaches for Markov parameter estimation, and in the latter, we analyze the multiple shooting approach. We demonstrate the limitations of the optimization problems posed by these existing methods by showing that they can be seen as special cases of the proposed optimization objective under certain simplifying assumptions: conditional independence of data and zero model error. Furthermore, we observe that our proposed approach has improved smoothness and inherent regularization that make it well-suited for system ID and provide mathematical explanations for these characteristics' origins. Finally, numerical simulations demonstrate a mean squared error over 8.7 times lower compared to multiple shooting when data are noisy and/or sparse. Moreover, the proposed approach can identify accurate and generalizable models even when there are more parameters than data or when the underlying system exhibits chaotic behavior.
Channel-state-information-based localization in 5G networks has been a promising way to obtain highly accurate positions compared to previous communication networks. However, there is no unified and effective platform to support the research on 5G localization algorithms. This paper releases a link-level simulator for 5G localization, which can depict realistic physical behaviors of the 5G positioning signal transmission. Specifically, we first develop a simulation architecture considering more elaborate parameter configuration and physical-layer processing. The architecture supports the link modeling at sub-6GHz and millimeter-wave (mmWave) frequency bands. Subsequently, the critical physical-layer components that determine the localization performance are designed and integrated. In particular, a lightweight new-radio channel model and hardware impairment functions that significantly limit the parameter estimation accuracy are developed. Finally, we present three application cases to evaluate the simulator, i.e. two-dimensional mobile terminal localization, mmWave beam sweeping, and beamforming-based angle estimation. The numerical results in the application cases present the performance diversity of localization algorithms in various impairment conditions.
The estimation law of unknown parameters vector ${\theta}$ is proposed for one class of nonlinearly parametrized regression equations $y\left( t \right) = \Omega \left( t \right)\Theta \left( \theta \right)$. We restrict our attention to parametrizations that are widely obtained in practical scenarios when polynomials in $\theta$ are used to form $\Theta \left( \theta \right)$. For them we introduce a new 'linearizability' assumption that a mapping from overparametrized vector of parameters $\Theta \left( \theta \right)$ to original one $\theta$ exists in terms of standard algebraic functions. Under such assumption and weak requirement of the regressor finite excitation, on the basis of dynamic regressor extension and mixing technique we propose a procedure to reduce the nonlinear regression equation to the linear parameterization without application of singularity causing operations and the need to identify the overparametrized parameters vector. As a result, an estimation law with exponential convergence rate is derived, which, unlike known solutions, (i) does not require a strict P-monotonicity condition to be met and a priori information about $\theta$ to be known, (ii) ensures elementwise monotonicity for the parameter error vector. The effectiveness of our approach is illustrated with both academic example and 2-DOF robot manipulator control problem.