We present a method for estimating parameters in stochastic models of biochemical reaction networks by fitting steady-state distributions using Wasserstein distances. We simulate a reaction network at different parameter settings and train a Gaussian process to learn the Wasserstein distance between observations and the simulator output for all parameters. We then use Bayesian optimization to find parameters minimizing this distance based on the trained Gaussian process. The effectiveness of our method is demonstrated on the three-stage model of gene expression and a genetic feedback loop for which moment-based methods are known to perform poorly. Our method is applicable to any simulator model of stochastic reaction networks, including Brownian Dynamics.
Mathematical models can provide quantitative insight into immunoreceptor signaling, but require parameterization and uncertainty quantification before making reliable predictions. We review currently available methods and software tools to address these problems. We consider gradient-based and gradient-free methods for point estimation of parameter values, and methods of profile likelihood, bootstrapping, and Bayesian inference for uncertainty quantification. We consider recent and potential future applications of these methods to systems-level modeling of immune-related phenomena.
In many practical parameter estimation problems, such as coefficient estimation of polynomial regression, the true model is unknown and thus, a model selection step is performed prior to estimation. The data-based model selection step affects the subsequent estimation. In particular, the oracle Cram\'er-Rao bound (CRB), which is based on knowledge of the true model, is inappropriate for post-model-selection performance analysis and system design outside the asymptotic region. In this paper, we investigate post-model-selection parameter estimation of a vector with an unknown support set, where this support set represents the model. We analyze the estimation performance of coherent estimators that force unselected parameters to zero. We use the mean-squared-selected-error (MSSE) criterion and introduce the concept of selective unbiasedness in the sense of Lehmann unbiasedness. We derive a non-Bayesian Cram\'er-Raotype bound on the MSSE and on the mean-squared-error (MSE) of any coherent estimator with a specific selective-bias function in the Lehmann sense. We implement the selective CRB for the special case of sparse vector estimation with an unknown support set. Finally, we demonstrate in simulations that the proposed selective CRB is an informative lower bound on the performance of the maximum selected likelihood estimator for a general linear model with the generalized information criterion and for sparse vector estimation with one step thresholding. It is shown that for these cases the selective CRB outperforms the oracle CRB and Sando-Mitra-Stoica CRB (SMS-CRB) [1].
A moving target may enter a radar coverage area unannounced and leave after an unspecified period, which implies that the target's entry time and departure time are unknown. In the absence of these time information, target detection and parameter estimation (DAPE) will be severely impacted. In this paper, we consider the coherent detection and parameters estimation problem for a radar moving target with unknown entry time and departure time (that is, the time when the target appears-in/leaves the radar detection field is unknown), involving across range cell (ARC) and Doppler spread (DS) effects within the observation period. A new algorithm, known as window Radon Fractional Fourier transform (WRFRFT) is proposed to detect and estimate the target's time parameters (i.e., entry time and departure time) and motion parameters (i.e., range, velocity and acceleration). The observation values of a maneuvering target are first intercepted and extracted by the window function and searching along the motion trajectory. Then these values are fractional Fourier transformed and well accumulated in the WRFRFT domain, where the DAPE of target could be accomplished thereafter. Experiments with simulated and real radar data sets prove its effectiveness.
This tutorial provides a worked example of using Dynamic Causal Modelling (DCM) and Parametric Empirical Bayes (PEB) to characterise inter-subject variability in neural circuitry (effective connectivity). This involves specifying a hierarchical model with two or more levels. At the first level, state space models (DCMs) are used to infer the effective connectivity that best explains a subject's neuroimaging timeseries (e.g. fMRI, MEG, EEG). Subject-specific connectivity parameters are then taken to the group level, where they are modelled using a General Linear Model (GLM) that partitions between-subject variability into designed effects and additive random effects. The ensuing (Bayesian) hierarchical model conveys both the estimated connection strengths and their uncertainty (i.e., posterior covariance) from the subject to the group level; enabling hypotheses to be tested about the commonalities and differences across subjects. This approach can also finesse parameter estimation at the subject level, by using the group-level parameters as empirical priors. We walk through this approach in detail, using data from a published fMRI experiment that characterised individual differences in hemispheric lateralization in a semantic processing task. The preliminary subject specific DCM analysis is covered in detail in a companion paper. This tutorial is accompanied by the example dataset and step-by-step instructions to reproduce the analyses.
Filtered backprojection (FBP) algorithm is a popular choice for complicated trajectory SAR image formation processing due to its inherent nonlinear motion compensation capability. However, how to efficiently autofocus the defocused FBP imagery when the motion measurement is not accurate enough is still a challenging problem. In this paper, a new interpretation of the FBP derivation is presented from the Fourier transform point of view. Based on this new viewpoint, the property of the residual 2-D phase error in FBP imagery is analyzed in detail. Then, by incorporating the derived a priori knowledge on the 2-D phase error, an accurate and efficient 2-D autofocus approach is proposed. The new approach performs the parameter estimation in a dimension-reduced parameter subspace by exploiting the a priori analytical structure of the 2-D phase error, therefore possesses much higher accuracy and efficiency than conventional blind methods. Finally, experimental results clearly demonstrate the effectiveness and robustness of the proposed method.
Millimeter wave (mm-wave) communication with large-scale antenna array configuration is seen as the key enabler of the next generation communication systems. Accurate knowledge of the mm-wave propagation channels is fundamental and essential. In this contribution, a novel complexity-efficient high resolution parameter estimation (HRPE) algorithm is proposed for the mm-wave channel with large-scale uniform circular array (UCA) applied. The proposed algorithm is able to obtain the high-resolution estimation results of the spherical channel propagation parameters. The prior channel information in the delay domain, i.e., the delay trajectories of individual propagation paths observed across the array elements, is exploited, by combining the high-resolution estimation principle and the phase mode excitation technique. Fast initializations, effective interference cancellations and reduced searching spaces achieved by the proposed schemes significantly decrease the algorithm complexity. Furthermore, the channel spatial non-stationarity in path gain across the array elements is considered for the first time in the literature for propagation parameter estimation, which is beneficial to obtain more realistic results as well as to decrease the complexity. A mm-wave measurement campaign at the frequency band of 28-30 GHz using a large-scale UCA is exploited to demonstrate and validate the proposed HRPE algorithm.
In this paper a new method is introduced in order to modify identification process of a gas power plant using a metaheuristic algorithm named Cuckoo Search (CS). Simulations play a significant role in dynamic analyses of power plants. This paper points out to a practical approach in model selection and parameter estimation of gas power plants. The identification and validation process concentrates on two subsystems: governor-turbine and exciter. Standard models GGOV1 and STB6 are preferred for the dynamical structures of governor-turbine and exciter respectively. Considering definite standard structure, main parameters of dynamical model are pre estimated via system identification methods based on field data. Then obtained parameters are tuned carefully using an iterative Cuckoo algorithm. Models must be validated by results derived via a trial and error series of simulation in comparison to measured test data. The procedure gradually yields in a valid model with precise estimated parameters. Simulation results show accuracy of identified models. Besides, a whiteness analysis has been performed in order to show the authenticity of the proposed method in another way. Despite various detailed models, practical attempts of model selection, identification, and validation in a real gas unit could rarely be found among literature. In this paper, Chabahar power plant in Iran, with total install capacity of 320 MW, is chosen as a benchmark for model validation.
When stock prices are observed at high frequencies, more information can be utilized in estimation of parameters of the price process. However, high-frequency data are contaminated by the market microstructure noise which causes significant bias in parameter estimation when not taken into account. We propose an estimator of the Ornstein-Uhlenbeck process based on the maximum likelihood which is robust to the noise and utilizes irregularly spaced data. We also show that the Ornstein-Uhlenbeck process contaminated by the independent Gaussian white noise and observed at discrete equidistant times follows an ARMA(1,1) process. To illustrate benefits of the proposed noise-robust approach, we introduce a novel intraday pairs trading strategy based on the mean-variance optimization. In an empirical study of 7 Big Oil companies, we show that the use of the proposed estimator of the Ornstein-Uhlenbeck process leads to an increase in profitability of the pairs trading strategy.
Affine jump-diffusions constitute a large class of continuous-time stochastic models that are particularly popular in finance and economics due to their analytical tractability. Methods for parameter estimation for such processes require ergodicity in order establish consistency and asymptotic normality of the associated estimators. In this paper, we develop stochastic stability conditions for affine jump-diffusions, thereby providing the needed large-sample theoretical support for estimating such processes. We establish ergodicity for such models by imposing a `strong mean reversion' condition and a mild condition on the distribution of the jumps, i.e. the finiteness of a logarithmic moment. Exponential ergodicity holds if the jumps have a finite moment of a positive order. In addition, we prove strong laws of large numbers and functional central limit theorems for additive functionals for this class of models.
Fifth generation (5G) cellular standards are set to utilize millimeter wave (mmWave) frequencies, which enable data speeds greater than 10 Gbps and sub-centimeter localization accuracy. These capabilities rely on accurate estimates of the channel parameters, which we define as the angle of arrival, angle of departure, and path distance for each path between the transmitter and receiver. Estimating the channel parameters in a computationally efficient manner poses a challenge because it requires estimation of parameters from a high-dimensional measurement -- particularly for multi-carrier systems since each subcarrier must be estimated separately. Additionally, channel parameter estimation must be able to handle hybrid beamforming, which uses a combination of digital and analog beamforming to reduce the number of required analog to digital converters. This paper introduces a channel parameter estimation technique based on the multilinear singular value decomposition (MSVD), a tensor analog of the singular value decomposition, for massive multiple input multiple output (MIMO) multi-carrier systems with hybrid beamforming. The MSVD tensor estimation approach provides a computationally efficient method and is shown to closely match the Cramer-Rao bound (CRB) of parameter estimates through simulations. Limitations of channel parameter estimation and communication waveform effects are also studied.
This paper deals with parameter estimation from extreme measurements. While being a special case of parameter estimation from partial data, in scenarios where only one sample from a given set of K measurements can be extracted, choosing only the minimum or the maximum (i.e., extreme) value from that set is of special interest because of the ultra-low energy, storage, and processing power required to extract extreme values from a given data set. We present a new methodology to analyze the performance of parameter estimation from extreme measurements. In particular, we present a general close-form approximation for the Cramer-Rao Lower Bound on the parameter estimation error, based on extreme values. We demonstrate our methodology on the case where the original measurements are exponential distributed, which is related to many practical applications. The analysis shows that the maximum values carry most of the information about the parameter of interest and that the additional information in the minimum is negligible. Moreover, it shows that for small sets of iid measurements (e.g. K=15) the use of the maximum can provide data compression with factor 15 while keeping about 50% of the information stored in the complete set.
The complexity of mathematical models describing respiratory mechanics has grown in recent years to integrate with cardiovascular models and incorporate nonlinear dynamics. However, additional model complexity has rarely been studied in the context of patient-specific observable data. This study investigates parameter identification of a previously developed nonlinear respiratory mechanics model (Ellwein Fix, PLoS ONE 2018) tuned to the physiology of 1 kg preterm infant, using local deterministic sensitivity analysis, subset selection, and gradient-based optimization. The model consists of 4 differential state equations with 31 parameters to predict airflow and dynamic pulmonary volumes and pressures generated under six simulation conditions. The relative sensitivity solutions of the model state equations with respect to each of the parameters were calculated with finite differences and a sensitivity ranking was created for each parameter and simulation. Subset selection identified a set of independent parameters that could be estimated for all six simulations. The combination of these analyses produced a subset of 6 independent sensitive parameters that could be estimated given idealized clinical data. All optimizations performed using pseudo-data with perturbed nominal parameters converged within 40 iterations and estimated parameters within ~8% of nominal values on average. This analysis indicates the feasibility of performing parameter estimation on real patient-specific data set described by a nonlinear respiratory mechanics model for studying dynamics in preterm infants.
Parameter estimation is a major challenge in computational modeling of biological processes. This is especially the case in image-based modeling where the inherently quantitative output of the model is measured against image data, which is typically noisy and non-quantitative. In addition, these models can have a high computational cost, limiting the number of feasible simulations, and therefore rendering most traditional parameter estimation methods unsuitable. In this paper, we present a pipeline that uses Gaussian process learning to estimate biological parameters from noisy, non-quantitative image data when the model has a high computational cost. This approach is first successfully tested on a parametric function with the goal of retrieving the original parameters. We then apply it to estimating parameters in a biological setting by fitting artificial in-situ hybridization (ISH) data of the developing murine limb bud. We expect that this method will be of use in a variety of modeling scenarios where quantitative data is missing and the use of standard parameter estimation approaches in biological modeling is prohibited by the computational cost of the model.
We construct a statistical indicator for the detection of short-term asset price bubbles based on the information content of bid and ask market quotes for plain vanilla put and call options. Our construction makes use of the martingale theory of asset price bubbles and the fact that such scenarios where the price for an asset exceeds its fundamental value can in principle be detected by analysis of the asymptotic behavior of the implied volatility surface. For extrapolating this implied volatility, we choose the SABR model, mainly because of its decent fit to real option market quotes for a broad range of maturities and its ease of calibration. As main theoretical result, we show that under lognormal SABR dynamics, we can compute a simple yet powerful closed-form martingale defect indicator by solving an ill-posed inverse calibration problem. In order to cope with the ill-posedness and to quantify the uncertainty which is inherent to such an indicator, we adopt a Bayesian statistical parameter estimation perspective. We probe the resulting posterior densities with a combination of optimization and adaptive Markov chain Monte Carlo methods, thus providing a full-blown uncertainty estimation of all the underlying parameters and the martingale defect indicator. Finally, we provide real-market tests of the proposed option-based indicator with focus on tech stocks due to increasing concerns about a tech bubble 2.0.
This paper investigates the impact of array switching patterns on the accuracy of parameter estimation of multipath components for a time division multiplexed (TDM) channel sounder. To measure fast time-varying channels, the conventional uniform array switching pattern poses a fundamental limit of the number of antennas that a TDM channel sounder can utilize. We propose a method, which is based on the simulated annealing algorithm, to find non-uniform array switching patterns for realistic antenna arrays, so that we can extend the Doppler estimation range of the channel sounder by suppressing the high sidelobes in the spatio-temporal ambiguity function. Monte Carlo simulations demonstrate that the optimal switching sequence leads to significantly smaller root mean square errors of both direction of departure and Doppler. Results can be applied in both vehicle-to-vehicle and mobile millimeter wave MIMO channel measurements.
Supervision of the physical layer of optical networks is an extremely relevant subject. To detect fiber faults, single-ended solutions such as the Optical Time Domain Reflectometer (OTDR) allow for precise measurements of fault profiles. Combining the OTDR with a signal processing approach for high-dimensional sparse parameter estimation allows for automated and reliable results in reduced time. In this work, a measurement system composed of a Photon-Counting OTDR data acquisition unit and a processing unit based on a Linearized Bregman Iterations algorithm for automatic fault finding is proposed. An in-depth comparative study of the proposed algorithm's fault-finding prowess in the presence of noise is presented. Characteristics such as sensitivity, specificity, processing time, and complexity, are analysed in simulated environments. Real-life measurements that are conducted using the Photon-Counting OTDR subsystem for data acquisition and the Linearized Bregman-based processing unit for automated data analysis demonstrated accurate results. It is concluded that the proposed measurement system is particularly well suited to the task of fault finding. The natural characteristic of the algorithm fosters embedding the solution in digital hardware, allowing for reduced costs and processing time.
A large number of recent genome-wide association studies (GWASs) for complex phenotypes confirm the early conjecture for polygenicity, suggesting the presence of large number of variants with only tiny or moderate effects. However, due to the limited sample size of a single GWAS, many associated genetic variants are too weak to achieve the genome-wide significance. These undiscovered variants further limit the prediction capability of GWAS. Restricted access to the individual-level data and the increasing availability of the published GWAS results motivate the development of methods integrating both the individual-level and summary-level data. How to build the connection between the individual-level and summary-level data determines the efficiency of using the existing abundant summary-level resources with limited individual-level data, and this issue inspires more efforts in the existing area. In this study, we propose a novel statistical approach, LEP, which provides a novel way of modeling the connection between the individual-level data and summary-level data. LEP integrates both types of data by \underline{LE}veraing \underline{P}leiotropy to increase the statistical power of risk variants identification and the accuracy of risk prediction. The algorithm for parameter estimation is developed to handle genome-wide-scale data. Through comprehensive simulation studies, we demonstrated the advantages of LEP over the existing methods. We further applied LEP to perform integrative analysis of Crohn's disease from WTCCC and summary statistics from GWAS of some other diseases, such as Type 1 diabetes, Ulcerative colitis and Primary biliary cirrhosis. LEP was able to significantly increase the statistical power of identifying risk variants and improve the risk prediction accuracy from 63.39\% ($\pm$ 0.58\%) to 68.33\% ($\pm$ 0.32\%) using about 195,000 variants.
In this paper, the parameter estimation problem for a multi-timescale adaptive threshold (MAT) neuronal model is investigated. By manipulating the system dynamics, which comprise of a non-resetting leaky integrator coupled with an adaptive threshold, the threshold voltage can be obtained as a realizable model that is linear in the unknown parameters. This linearly parametrized realizable model is then utilized inside a prediction error based framework to identify the threshold parameters with the purpose of predicting single neuron precise firing times. The iterative linear least squares estimation scheme is evaluated using both synthetic data obtained from an exact model as well as experimental data obtained from in vitro rat somatosensory cortical neurons. Results show the ability of this approach to fit the MAT model to different types of fluctuating reference data. The performance of the proposed approach is seen to be superior when comparing with existing identification approaches used by the neuronal community.
Models of biological systems often have many unknown parameters that must be determined in order for model behavior to match experimental observations. Commonly-used methods for parameter estimation that return point estimates of the best-fit parameters are insufficient when models are high dimensional and under-constrained. As a result, Bayesian methods, which treat model parameters as random variables and attempt to estimate their probability distributions given data, have become popular in systems biology. Bayesian parameter estimation often relies on Markov Chain Monte Carlo (MCMC) methods to sample model parameter distributions, but the slow convergence of MCMC sampling can be a major bottleneck. One approach to improving performance is parallel tempering (PT), a physics-based method that uses swapping between multiple Markov chains run in parallel at different temperatures to accelerate sampling. The temperature of a Markov chain determines the probability of accepting an unfavorable move, so swapping with higher temperatures chains enables the sampling chain to escape from local minima. In this work we compared the MCMC performance of PT and the commonly-used Metropolis-Hastings (MH) algorithm on six biological models of varying complexity. We found that for simpler models PT accelerated convergence and sampling, and that for more complex models, PT often converged in cases MH became trapped in non-optimal local minima. We also developed a freely-available MATLAB package for Bayesian parameter estimation called PTempEst (http://github.com/RuleWorld/ptempest), which is closely integrated with the popular BioNetGen software for rule-based modeling of biological systems.