This is the second part of the two-part paper considering the communications under the bursty mixed noise composed of white Gaussian noise and colored non-Gaussian impulsive noise. In the first part, based on Gaussian distribution and student distribution, we proposed a multivariate bursty mixed noise model and designed model parameter estimation algorithms. However, the performance of a communication system will significantly deteriorate under the bursty mixed noise if a conventional signal detection algorithm with respect to Gaussian noise is applied. To address this issue, in the second part, we leverage the probability density function (PDF) to derive the maximum likelihood (ML) demodulation methods for both linear and nonlinear modulations, including M-array PSK (M-PSK) and MSK modulation schemes. We analyze the theoretical bit error rate (BER) performance of M-PSK and present close-form BER expressions. For the MSK demodulation based on the Viterbi algorithm, we derive a lower and upper bound of BER. Simulation results showcase that the proposed demodulation methods outperform baselines by more than 2.5dB when the BER performance reaches the order of magnitude of $10^{-3}$, and the theoretical analysis matches the simulated results well.
Soft sensing is a way to indirectly obtain information of signals for which direct sensing is difficult or prohibitively expensive. It may not \textit{a priori} be evident which sensors provide useful information about the target signal, and various operating conditions often necessitate different models. In this paper, we provide a systematic method to construct a soft sensor that can deal with these issues. We propose a single estimation criterion, where the objectives are encoded in terms of model fit, model sparsity (reducing the number of different models), and model parameter coefficient sparsity (to exclude irrelevant sensors). The proposed method is tested on real-world scenarios involving prototype vehicles, demonstrating its effectiveness.
Biomanufacturing innovation relies on an efficient Design of Experiments (DoEs) to optimize processes and product quality. Traditional DoE methods, ignoring the underlying bioprocessing mechanisms, often suffer from a lack of interpretability and sample efficiency. This limitation motivates us to create a new optimal learning approach for digital twin model calibration. In this study, we consider the cell culture process multi-scale mechanistic model, also known as Biological System-of-Systems (Bio-SoS). This model with a modular design, composed of sub-models, allows us to integrate data across various production processes. To calibrate the Bio-SoS digital twin, we evaluate the mean squared error of model prediction and develop a computational approach to quantify the impact of parameter estimation error of individual sub-models on the prediction accuracy of digital twin, which can guide sample-efficient and interpretable DoEs.
In this paper, we present a novel approach to accelerate the Bayesian inference process, focusing specifically on the nested sampling algorithms. Bayesian inference plays a crucial role in cosmological parameter estimation, providing a robust framework for extracting theoretical insights from observational data. However, its computational demands can be substantial, primarily due to the need for numerous likelihood function evaluations. Our proposed method utilizes the power of deep learning, employing feedforward neural networks to approximate the likelihood function dynamically during the Bayesian inference process. Unlike traditional approaches, our method trains neural networks on-the-fly using the current set of live points as training data, without the need for pre-training. This flexibility enables adaptation to various theoretical models and datasets. We perform simple hyperparameter optimization using genetic algorithms to suggest initial neural network architectures for learning each likelihood function. Once sufficient accuracy is achieved, the neural network replaces the original likelihood function. The implementation integrates with nested sampling algorithms and has been thoroughly evaluated using both simple cosmological dark energy models and diverse observational datasets. Additionally, we explore the potential of genetic algorithms for generating initial live points within nested sampling inference, opening up new avenues for enhancing the efficiency and effectiveness of Bayesian inference methods.
We consider the problem of learning uncertainty regions for parameter estimation problems. The regions are ellipsoids that minimize the average volumes subject to a prescribed coverage probability. As expected, under the assumption of jointly Gaussian data, we prove that the optimal ellipsoid is centered around the conditional mean and shaped as the conditional covariance matrix. In more practical cases, we propose a differentiable optimization approach for approximately computing the optimal ellipsoids using a neural network with proper calibration. Compared to existing methods, our network requires less storage and less computations in inference time, leading to accurate yet smaller ellipsoids. We demonstrate these advantages on four real-world localization datasets.
Deep learning has emerged as a promising approach for learning the nonlinear mapping between diffusion-weighted MR images and tissue parameters, which enables automatic and deep understanding of the brain microstructures. However, the efficiency and accuracy in estimating multiple microstructural parameters derived from multiple diffusion models are still limited since previous studies tend to estimate parameter maps from distinct models with isolated signal modeling and dense sampling. This paper proposes DeepMpMRI, an efficient framework for fast and high-fidelity multiple microstructural parameter estimation from multiple models using highly sparse sampled q-space data. DeepMpMRI is equipped with a newly designed tensor-decomposition-based regularizer to effectively capture fine details by exploiting the high-dimensional correlation across microstructural parameters. In addition, we introduce a Nesterov-based adaptive learning algorithm that optimizes the regularization parameter dynamically to enhance the performance. DeepMpMRI is an extendable framework capable of incorporating flexible network architecture. Experimental results on the HCP dataset and the Alzheimer's disease dataset both demonstrate the superiority of our approach over 5 state-of-the-art methods in simultaneously estimating multi-model microstructural parameter maps for DKI and NODDI model with fine-grained details both quantitatively and qualitatively, achieving 4.5 - 15 $\times$ acceleration compared to the dense sampling of a total of 270 diffusion gradients.
Is it possible to make online decisions when personalized covariates are unavailable? We take a collaborative-filtering approach for decision-making based on collective preferences. By assuming low-dimensional latent features, we formulate the covariate-free decision-making problem as a matrix completion bandit. We propose a policy learning procedure that combines an $\varepsilon$-greedy policy for decision-making with an online gradient descent algorithm for bandit parameter estimation. Our novel two-phase design balances policy learning accuracy and regret performance. For policy inference, we develop an online debiasing method based on inverse propensity weighting and establish its asymptotic normality. Our methods are applied to data from the San Francisco parking pricing project, revealing intriguing discoveries and outperforming the benchmark policy.
Recent studies indicate that the noise characteristics of phasor measurement units (PMUs) can be more accurately described by non-Gaussian distributions. Consequently, estimation techniques based on Gaussian noise assumptions may produce poor results with PMU data. This paper considers the PMU based line parameter estimation (LPE) problem, and investigates the performance of four state-of-the-art techniques in solving this problem in presence of non-Gaussian measurement noise. The rigorous comparative analysis highlights the merits and demerits of each technique w.r.t. the LPE problem, and identifies conditions under which they are expected to give good results.
This work introduces a new method for selecting the number of components in finite mixture models (FMMs) using variational Bayes, inspired by the large-sample properties of the Evidence Lower Bound (ELBO) derived from mean-field (MF) variational approximation. Specifically, we establish matching upper and lower bounds for the ELBO without assuming conjugate priors, suggesting the consistency of model selection for FMMs based on maximizing the ELBO. As a by-product of our proof, we demonstrate that the MF approximation inherits the stable behavior (benefited from model singularity) of the posterior distribution, which tends to eliminate the extra components under model misspecification where the number of mixture components is over-specified. This stable behavior also leads to the $n^{-1/2}$ convergence rate for parameter estimation, up to a logarithmic factor, under this model overspecification. Empirical experiments are conducted to validate our theoretical findings and compare with other state-of-the-art methods for selecting the number of components in FMMs.
The data-driven newsvendor problem with features has recently emerged as a significant area of research, driven by the proliferation of data across various sectors such as retail, supply chains, e-commerce, and healthcare. Given the sensitive nature of customer or organizational data often used in feature-based analysis, it is crucial to ensure individual privacy to uphold trust and confidence. Despite its importance, privacy preservation in the context of inventory planning remains unexplored. A key challenge is the nonsmoothness of the newsvendor loss function, which sets it apart from existing work on privacy-preserving algorithms in other settings. This paper introduces a novel approach to estimate a privacy-preserving optimal inventory policy within the f-differential privacy framework, an extension of the classical $(\epsilon, \delta)$-differential privacy with several appealing properties. We develop a clipped noisy gradient descent algorithm based on convolution smoothing for optimal inventory estimation to simultaneously address three main challenges: (1) unknown demand distribution and nonsmooth loss function; (2) provable privacy guarantees for individual-level data; and (3) desirable statistical precision. We derive finite-sample high-probability bounds for optimal policy parameter estimation and regret analysis. By leveraging the structure of the newsvendor problem, we attain a faster excess population risk bound compared to that obtained from an indiscriminate application of existing results for general nonsmooth convex loss. Our bound aligns with that for strongly convex and smooth loss function. Our numerical experiments demonstrate that the proposed new method can achieve desirable privacy protection with a marginal increase in cost.
Differential equations are pivotal in modeling and understanding the dynamics of various systems, offering insights into their future states through parameter estimation fitted to time series data. In fields such as economy, politics, and biology, the observation data points in the time series are often independently obtained (i.e., Repeated Cross-Sectional (RCS) data). With RCS data, we found that traditional methods for parameter estimation in differential equations, such as using mean values of time trajectories or Gaussian Process-based trajectory generation, have limitations in estimating the shape of parameter distributions, often leading to a significant loss of data information. To address this issue, we introduce a novel method, Estimation of Parameter Distribution (EPD), providing accurate distribution of parameters without loss of data information. EPD operates in three main steps: generating synthetic time trajectories by randomly selecting observed values at each time point, estimating parameters of a differential equation that minimize the discrepancy between these trajectories and the true solution of the equation, and selecting the parameters depending on the scale of discrepancy. We then evaluated the performance of EPD across several models, including exponential growth, logistic population models, and target cell-limited models with delayed virus production, demonstrating its superiority in capturing the shape of parameter distributions. Furthermore, we applied EPD to real-world datasets, capturing various shapes of parameter distributions rather than a normal distribution. These results effectively address the heterogeneity within systems, marking a substantial progression in accurately modeling systems using RCS data.
The purpose of this paper is twofold. First, we propose a novel algorithm for estimating parameters in one-dimensional Gaussian mixture models (GMMs). The algorithm takes advantage of the Hankel structure inherent in the Fourier data obtained from independent and identically distributed (i.i.d) samples of the mixture. For GMMs with a unified variance, a singular value ratio functional using the Fourier data is introduced and used to resolve the variance and component number simultaneously. The consistency of the estimator is derived. Compared to classic algorithms such as the method of moments and the maximum likelihood method, the proposed algorithm does not require prior knowledge of the number of Gaussian components or good initial guesses. Numerical experiments demonstrate its superior performance in estimation accuracy and computational cost. Second, we reveal that there exists a fundamental limit to the problem of estimating the number of Gaussian components or model order in the mixture model if the number of i.i.d samples is finite. For the case of a single variance, we show that the model order can be successfully estimated only if the minimum separation distance between the component means exceeds a certain threshold value and can fail if below. We derive a lower bound for this threshold value, referred to as the computational resolution limit, in terms of the number of i.i.d samples, the variance, and the number of Gaussian components. Numerical experiments confirm this phase transition phenomenon in estimating the model order. Moreover, we demonstrate that our algorithm achieves better scores in likelihood, AIC, and BIC when compared to the EM algorithm.
Traditionally, batch least squares (BLS) and recursive least squares (RLS) are used for identification of a vector of parameters that form a linear model. In some situations, however, it is of interest to identify parameters in a matrix structure. In this case, a common approach is to transform the problem into standard vector form using the vectorization (vec) operator and the Kronecker product, known as vec-permutation. However, the use of the Kronecker product introduces extraneous zero terms in the regressor, resulting in unnecessary additional computational and space requirements. This work derives matrix BLS and RLS formulations which, under mild assumptions, minimize the same cost as the vec-permutation approach. This new approach requires less computational complexity and space complexity than vec-permutation in both BLS and RLS identification. It is also shown that persistent excitation guarantees convergence to the true matrix parameters. This method can used to improve computation time in the online identification of multiple-input, multiple-output systems for indirect adaptive model predictive control.
This paper presents subspace of information forgetting recursive least squares (SIFt-RLS), a directional forgetting algorithm which, at each step, forgets only in row space of the regressor matrix, or the \textit{information subspace}. As a result, SIFt-RLS tracks parameters that are in excited directions while not changing parameter estimation in unexcited directions. It is shown that SIFt-RLS guarantees an upper and lower bound of the covariance matrix, without assumptions of persistent excitation, and explicit bounds are given. Furthermore, sufficient conditions are given for the uniform Lyapunov stability and global uniform exponential stability of parameter estimation error in SIFt-RLS when estimating fixed parameters without noise. SIFt-RLS is compared to other RLS algorithms from the literature in a numerical example without persistently exciting data.
Integrated super-resolution sensing and communication (ISSAC) is a promising technology to achieve extremely high sensing performance for critical parameters, such as the angles of the wireless channels. In this paper, we propose an ISSAC-based channel estimation method, which requires little or even no pilot, yet still achieves accurate channel state information (CSI) estimation. The key idea is to exploit the fact that subspace-based super-resolution algorithms such as multiple signal classification (MUSIC) do not require a priori known pilots for accurate parameter estimation. Therefore, in the proposed method, the angles of the multi-path channel components are first estimated in a pilot-free manner while communication data symbols are sent. After that, the multi-path channel coefficients are estimated, where very little pilots are needed. The reasons are two folds. First, compared to the conventional channel estimation methods purely relying on channel training, much fewer parameters need to be estimated once the multi-path angles are accurately estimated. Besides, with angles obtained, the beamforming gain is also enjoyed when pilots are sent to estimate the channel path gains. To rigorously study the performance of the proposed method, we first consider the basic line-of-sight (LoS) channel. By analyzing the minimum mean square error (MMSE) of channel estimation and the resulting beamforming gains, we show that our proposed method significantly outperforms the conventional methods purely based on channel training. We then extend the study to the more general multipath channels. Simulation results are provided to demonstrate our theoretical results.
We consider the Ornstein-Uhlenbeck (OU) process, a stochastic process widely used in finance, physics, and biology. Parameter estimation of the OU process is a challenging problem. Thus, we review traditional tracking methods and compare them with novel applications of deep learning to estimate the parameters of the OU process. We use a multi-layer perceptron to estimate the parameters of the OU process and compare its performance with traditional parameter estimation methods, such as the Kalman filter and maximum likelihood estimation. We find that the multi-layer perceptron can accurately estimate the parameters of the OU process given a large dataset of observed trajectories and, on average, outperforms traditional parameter estimation methods.
This article introduces differentially private log-location-scale (DP-LLS) regression models, which incorporate differential privacy into LLS regression through the functional mechanism. The proposed models are established by injecting noise into the log-likelihood function of LLS regression for perturbed parameter estimation. We will derive the sensitivities utilized to determine the magnitude of the injected noise and prove that the proposed DP-LLS models satisfy $\epsilon$-differential privacy. In addition, we will conduct simulations and case studies to evaluate the performance of the proposed models. The findings suggest that predictor dimension, training sample size, and privacy budget are three key factors impacting the performance of the proposed DP-LLS regression models. Moreover, the results indicate that a sufficiently large training dataset is needed to simultaneously ensure decent performance of the proposed models and achieve a satisfactory level of privacy protection.
Breast cancer is one of the most challenging global health problems among women. This study investigates the intricate breast tumor microenvironment (TME) dynamics utilizing data from Mammary-specific Polyomavirus Middle T Antigen Overexpression mouse models (MMTV-PyMT). It incorporates Endothelial Cells (ECs), oxygen, and Vascular Endothelial Growth Factors (VEGF) to examine the interplay of angiogenesis, hypoxia, VEGF, and the immune cells in cancer progression. We introduce an approach to impute the immune cell fractions within the TME using single-cell RNA-sequencing (scRNA-seq) data from MMTV-PyMT mice. We further quantify our analysis by estimating cell counts using cell size data and laboratory findings from existing literature. Parameter estimation is carried out via a Hybrid Genetic Algorithm (HGA). Our simulations reveal various TME behaviors, emphasizing the critical role of adipocytes, angiogenesis, hypoxia, and oxygen transport in driving immune responses and cancer progression. The global sensitivity analyses highlight potential therapeutic intervention points, such as VEGFs' critical role in EC growth and oxygen transportation and severe hypoxia's effect on the cancer and the total number of cells. The VEGF-mediated production rate of ECs shows an essential time-dependent impact, highlighting the importance of early intervention in slowing cancer progression. These findings align with the observations from the clinical trials demonstrating the efficacy of VEGF inhibitors and suggest a timely intervention for better outcomes.
One of the most crucial and lethal characteristics of solid tumors is represented by the increased ability of cancer cells to migrate and invade other organs during the so-called metastatic spread. This is allowed thanks to the production of matrix metalloproteinases (MMPs), enzymes capable of degrading a type of collagen abundant in the basal membrane separating the epithelial tissue from the connective one. In this work, we employ a synergistic experimental and mathematical modelling approach to explore the invasion process of tumor cells. A athematical model composed of reaction-diffusion equations describing the evolution of the tumor cells density on a gelatin substrate, MMPs enzymes concentration and the degradation of the gelatin is proposed. This is completed with a calibration strategy. We perform a sensitivity analysis and explore a parameter estimation technique both on synthetic and experimental data in order to find the optimal parameters that describe the in vitro experiments. A comparison between numerical and experimental solutions ends the work.
When fitting a multi-parameter model to a data set, computer algorithms may suggest that a range of parameters provide equally reasonable fits, making the parameter estimation difficult. Here, we prove this fact for an SIR model. We say a set of parameter values is a good fit to outbreak data if the solution has the data's three most significant characteristics: the standard deviation, the mean time, and the total number of cases. In our model, in addition to the "basic reproduction number" $R_0$, three other parameters need to be estimated to fit a solution to outbreak data. We will show that those parameters can be chosen so that each gives a linear transformation of a solution's incidence data. As a result, we show that for every choice of $R_0>1$, there is a good fit for each outbreak. We also illustrate our results by providing the least square best fits of the New York City and London data sets of the Omicron variant of COVID-19. Furthermore, we show how versions of the SIR model with $N$ compartments have far more good fits- - indeed a high dimensional set of good fits -- for each target -- showing that more complicated models may have an even greater problem in overparametrizing outbreak characteristics.