Mathematical models can provide quantitative insight into immunoreceptor signaling, but require parameterization and uncertainty quantification before making reliable predictions. We review currently available methods and software tools to address these problems. We consider gradient-based and gradient-free methods for point estimation of parameter values, and methods of profile likelihood, bootstrapping, and Bayesian inference for uncertainty quantification. We consider recent and potential future applications of these methods to systems-level modeling of immune-related phenomena.
Reinforcement learning (RL) methods have demonstrated their efficiency in simulation environments. However, many applications for which RL offers great potential, such as autonomous driving, are also safety critical and require a certified closed-loop behavior in order to meet safety specifications in the presence of physical constraints. This paper introduces a concept, called probabilistic model predictive safety certification (PMPSC), which can be combined with any RL algorithm and provides provable safety certificates in terms of state and input chance constraints for potentially large-scale systems. The certificate is realized through a stochastic tube that safely connects the current system state with a terminal set of states, that is known to be safe. A novel formulation in terms of a convex receding horizon problem allows a recursively feasible real-time computation of such probabilistic tubes, despite the presence of possibly unbounded disturbances. A design procedure for PMPSC relying on bayesian inference and recent advances in probabilistic set invariance is presented. Using a numerical car simulation, the method and its design procedure are illustrated by enhancing a simple RL algorithm with safety certificates.
In this study, Bayesian inference is developed for structural vector autoregressive models in which the structural parameters are identified via Markov-switching heteroskedasticity. In such a model, restrictions that are just-identifying in the homoskedastic case, become over-identifying and can be tested. A set of parametric restrictions is derived under which the structural matrix is globally or partially identified and a Savage-Dickey density ratio is used to assess the validity of the identification conditions. The latter is facilitated by analytical derivations that make the computations fast and numerical standard errors small. As an empirical example, monetary models are compared using heteroskedasticity as an additional device for identification. The empirical results support models with money in the interest rate reaction function.
Deep Neural Networks (DNNs) have aroused great attention in Compressed Sensing (CS) restoration. However, the working mechanism of DNNs is not explainable, thereby it is unclear that how to design an optimal DNNs for CS restoration. In this paper, we propose a novel statistical framework to explain DNNs, which proves that the hidden layers of DNNs are equivalent to Gibbs distributions and interprets DNNs as a Bayesian hierarchical model. The framework provides a Bayesian perspective to explain the working mechanism of DNNs, namely some hidden layers learn a prior distribution and other layers learn a likelihood distribution. Moreover, the framework provides insights into DNNs and reveals two inherent limitations of DNNs for CS restoration. In contrast to most previous works designing an end-to-end DNNs for CS restoration, we propose a novel DNNs to model a prior distribution only, which can circumvent the limitations of DNNs. Given the prior distribution generated from the DNNs, we design a Bayesian inference algorithm to realize CS restoration in the framework of Bayesian Compressed Sensing. Finally, extensive simulations validate the proposed theory of DNNs and demonstrate that the proposed algorithm outperforms the state-of-the-art CS restoration methods.
A method for implicit variable selection in mixture of experts frameworks is proposed. We introduce a prior structure where information is taken from a set of independent covariates. Robust class membership predictors are identified using a normal gamma prior. The resulting model setup is used in a finite mixture of Bernoulli distributions to find homogenous clusters of women in Mozambique based on their information sources on HIV. Fully Bayesian inference is carried out via the implementation of a Gibbs sampler.
Neural responses in the cortex change over time both systematically, due to ongoing plasticity and learning, and seemingly randomly, due to various sources of noise and variability. Most previous work considered each of these processes, learning and variability, in isolation -- here we study neural networks exhibiting both and show that their interaction leads to the emergence of powerful computational properties. We trained neural networks on classical unsupervised learning tasks, in which the objective was to represent their inputs in an efficient, easily decodable form, with an additional cost for neural reliability which we derived from basic biophysical considerations. This cost on reliability introduced a tradeoff between energetically cheap but inaccurate representations and energetically costly but accurate ones. Despite the learning tasks being non-probabilistic, the networks solved this tradeoff by developing a probabilistic representation: neural variability represented samples from statistically appropriate posterior distributions that would result from performing probabilistic inference over their inputs. We provide an analytical understanding of this result by revealing a connection between the cost of reliability, and the objective for a state-of-the-art Bayesian inference strategy: variational autoencoders. We show that the same cost leads to the emergence of increasingly accurate probabilistic representations as networks become more complex, from single-layer feed-forward, through multi-layer feed-forward, to recurrent architectures. Our results provide insights into why neural responses in sensory areas show signatures of sampling-based probabilistic representations, and may inform future deep learning algorithms and their implementation in stochastic low-precision computing systems.
Heterogeneous nature of the vehicular networks, which results from the co-existence of human-driven, semi-automated, and fully autonomous vehicles, is a challenging phenomenon toward the realization of the intelligent transportation systems with an acceptable level of safety, comfort, and efficiency. Safety applications highly suffer from communication resource limitations, specifically in dense and congested vehicular networks. The idea of model-based communication (MBC) has been recently proposed to address this issue. In this work, we propose Gaussian Process-based Stochastic Hybrid System with Cumulative Relevant History (CRH-GP-SHS) framework, which is a hierarchical stochastic hybrid modeling structure, built upon a non-parametric Bayesian inference method, i.e. Gaussian processes. This framework is proposed in order to be employed within the MBC context to jointly model driver/vehicle behavior as a stochastic object. Non-parametric Bayesian methods relieve the limitations imposed by non-evolutionary model structures and enable the proposed framework to properly capture different stochastic behaviors. The performance of the proposed CRH-GP-SHS framework at the inter-mode level has been evaluated over a set of realistic lane change maneuvers from NGSIM-US101 dataset. The results show a noticeable performance improvement for GP in comparison to the baseline constant speed model, specifically in critical situations such as highly congested networks. Moreover, an augmented model has also been proposed which is a composition of GP and constant speed models and capable of capturing the driver behavior under various network reliability conditions.
Hidden Markov models (HMMs) are popular time series model in many fields including ecology, economics and genetics. HMMs can be defined over discrete or continuous time, though here we only cover the former. In the field of movement ecology in particular, HMMs have become a popular tool for the analysis of movement data because of their ability to connect observed movement data to an underlying latent process, generally interpreted as the animal's unobserved behavior. Further, we model the tendency to persist in a given behavior over time. Notation presented here will generally follow the format of Zucchini et al. (2016) and cover HMMs applied in an unsupervised case to animal movement data, specifically positional data. We provide Stan code to analyze movement data of the wild haggis as presented first in Michelot et al. (2016).
Genetic sequence data of pathogens are increasingly used to investigate transmission dynamics in both endemic diseases and disease outbreaks; such research can aid in development of appropriate interventions and in design of studies to evaluate them. Several methods have been proposed to infer transmission chains from sequence data; however, existing methods do not generally reliably reconstruct transmission trees because genetic sequence data or inferred phylogenetic trees from such data are insufficient for accurate inference regarding transmission chains. In this paper, we demonstrate the lack of a one-to-one relationship between phylogenies and transmission trees, and also show that information regarding infection times together with genetic sequences permit accurate reconstruction of transmission trees. We propose a Bayesian inference method for this purpose and demonstrate that precision of inference regarding these transmission trees depends on precision of the estimated times of infection. We also illustrate the use of these methods to study features of epidemic dynamics, such as the relationship between characteristics of nodes and average number of outbound edges or inbound edges-- signifying possible transmission events from and to nodes. We study the performance of the proposed method in simulation experiments and demonstrate its superiority in comparison to an alternative method. We apply them to a transmission cluster in San Diego and investigate the impact of biological, behavioral, and demographic factors.
Since exchange economy considerably varies in the market assets, asset prices have become an attractive research area for investigating and modeling ambiguous and uncertain information in today markets. This paper proposes a new generative uncertainty mechanism based on the Bayesian Inference and Correntropy (BIC) technique for accurately evaluating asset pricing in markets. This technique examines the potential processes of risk, ambiguity, and variations of market information in a controllable manner. We apply the new BIC technique to a consumption asset-pricing model in which the consumption variations are modeled using the Bayesian network model with observing the dynamics of asset pricing phenomena in the data. These dynamics include the procyclical deviations of price, the countercyclical deviations of equity premia and equity volatility, the leverage impact and the mean reversion of excess returns. The key findings reveal that the precise modeling of asset information can estimate price changes in the market effectively.
Scalability is one of the major issues for real-world Vehicle-to-Vehicle network realization. To tackle this challenge, a stochastic hybrid modeling framework based on a non-parametric Bayesian inference method, i.e., hierarchical Dirichlet process (HDP), is investigated in this paper. This framework is able to jointly model driver/vehicle behavior through forecasting the vehicle dynamical time-series. This modeling framework could be merged with the notion of model-based information networking, which is recently proposed in the vehicular literature, to overcome the scalability challenges in dense vehicular networks via broadcasting the behavioral models instead of raw information dissemination. This modeling approach has been applied on several scenarios from the realistic Safety Pilot Model Deployment (SPMD) driving data set and the results show a higher performance of this model in comparison with the zero-hold method as the baseline.
Traditional theories of sensorimotor learning posit that animals use sensory error signals to find the optimal motor command in the face of Gaussian sensory and motor noise. However, most such theories cannot explain common behavioral observations, for example that smaller sensory errors are more readily corrected than larger errors and that large abrupt (but not gradually introduced) errors lead to weak learning. Here we propose a new theory of sensorimotor learning that explains these observations. The theory posits that the animal learns an entire probability distribution of motor commands rather than trying to arrive at a single optimal command, and that learning arises via Bayesian inference when new sensory information becomes available. We test this theory using data from a songbird, the Bengalese finch, that is adapting the pitch (fundamental frequency) of its song following perturbations of auditory feedback using miniature headphones. We observe the distribution of the sung pitches to have long, non-Gaussian tails, which, within our theory, explains the observed dynamics of learning. Further, the theory makes surprising predictions about the dynamics of the shape of the pitch distribution, which we confirm experimentally.
The accuracy of least squares calibration using option premiums and particle filtering of price data to find model parameters is determined. Derivative models using exponential L\'evy processes are calibrated using regularized weighted least squares with respect to the minimal entropy martingale measure. Sequential importance resampling is used for the Bayesian inference problem of time series parameter estimation with proposal distribution determined using extended Kalman filter. The algorithms converge to their respective global optima using a highly parallelizable statistical optimization approach using a grid of initial positions. Each of these methods should produce the same parameters. We investigate this assertion.
The aim of this paper was to develop statistical models to estimate individual breed composition based on the previously proposed idea of regressing discrete random variables corresponding to counts of reference alleles of biallelic molecular markers located across the genome on the allele frequencies of each marker in the pure (base) breeds. Some of the existing regression-based methods do not guarantee that estimators of breed composition will lie in the appropriate parameter space and none of them account for uncertainty about allele frequencies in the pure breeds, that is, uncertainty about the design matrix. In order to overcome these limitations, we proposed two Bayesian generalized linear models. For each individual, both models assume that the counts of the reference allele at each marker locus follow independent Binomial distributions, use the logit link, and pose a Dirichlet prior over the vector of regression coefficients (which corresponds to breed composition). This prior guarantees that point estimators of breed composition like the posterior mean pertain to the appropriate space. The difference between these models is that model termed BIBI does not account for uncertainty about the design matrix, while model termed BIBI2 accounts for such an uncertainty by assigning independent Beta priors to the entries of this matrix. We implemented these models in a multibreed Angus-Brahman population. Posterior means were used as point estimators of breed composition. In addition, the ordinary least squares estimator proposed by Kuehn et al. (2011) (OLSK) was also computed. BIBI and BIBI2 estimated breed composition more accurately than OLSK, and BIBI2 had an 8.3% improvement in accuracy as compared to BIBI.
Cohort effects are important factors in determining the evolution of human mortality for certain countries. Extensions of dynamic mortality models with cohort features have been proposed in the literature to account for these factors under the generalised linear modelling framework. In this paper we approach the problem of mortality modelling with cohort factors incorporated through a novel formulation under a state-space methodology. In the process we demonstrate that cohort factors can be formulated naturally under the state-space framework, despite the fact that cohort factors are indexed according to year-of-birth rather than year. Bayesian inference for cohort models in a state-space formulation is then developed based on an efficient Markov chain Monte Carlo sampler, allowing for the quantification of parameter uncertainty in cohort models and resulting mortality forecasts that are used for life expectancy and life table constructions. The effectiveness of our approach is examined through comprehensive empirical studies involving male and female populations from various countries. Our results show that cohort patterns are present for certain countries that we studied and the inclusion of cohort factors are crucial in capturing these phenomena, thus highlighting the benefits of introducing cohort models in the state-space framework. Forecasting of cohort models is also discussed in light of the projection of cohort factors.
Bayesian inference methods rely on numerical algorithms for both model selection and parameter inference. In general, these algorithms require a high computational effort to yield reliable estimates. One of the major challenges in phylogenetics is the estimation of the marginal likelihood. This quantity is commonly used for comparing different evolutionary models, but its calculation, even for simple models, incurs high computational cost. Another interesting challenge relates to the estimation of the posterior distribution. Often, long Markov chains are required to get sufficient samples to carry out parameter inference, especially for tree distributions. In general, these problems are addressed separately by using different procedures. Nested sampling (NS) is a Bayesian computation algorithm which provides the means to estimate marginal likelihoods together with their uncertainties, and to sample from the posterior distribution at no extra cost. The methods currently used in phylogenetics for marginal likelihood estimation lack in practicality due to their dependence on many tuning parameters and the inability of most implementations to provide a direct way to calculate the uncertainties associated with the estimates. To address these issues, we introduce NS to phylogenetics. Its performance is assessed under different scenarios and compared to established methods. We conclude that NS is a competitive and attractive algorithm for phylogenetic inference. An implementation is available as a package for BEAST 2 under the LGPL licence, accessible at https://github.com/BEAST2-Dev/nested-sampling.
Next generation sequencing allows the identification of genes consisting of differentially expressed transcripts, a term which usually refers to changes in the overall expression level. A specific type of differential expression is differential transcript usage (DTU) and targets changes in the relative within gene expression of a transcript. The contribution of this paper is to: (a) extend the use of cjBitSeq to the DTU context, a previously introduced Bayesian model which is originally designed for identifying changes in overall expression levels and (b) propose a Bayesian version of DRIMSeq, a frequentist model for inferring DTU. cjBitSeq is a read based model and performs fully Bayesian inference by MCMC sampling on the space of latent state of each transcript per gene. BayesDRIMSeq is a count based model and estimates the Bayes Factor of a DTU model against a null model using Laplace's approximation. The proposed models are benchmarked against the existing ones using a recent independent simulation study as well as a real RNA-seq dataset. Our results suggest that the Bayesian methods exhibit similar performance with DRIMSeq in terms of precision/recall but offer better calibration of False Discovery Rate.
We consider a continuous-time Markov chain model of SIR disease dynamics with two levels of mixing. For this so-called stochastic households model, we provide two methods for inferring the model parameters---governing within-household transmission, recovery, and between-household transmission---from data of the day upon which each individual became infectious and the household in which each infection occurred, as would be available from first few hundred studies. Each method is a form of Bayesian Markov Chain Monte Carlo that allows us to calculate a joint posterior distribution for all parameters and hence the household reproduction number and the early growth rate of the epidemic. The first method performs exact Bayesian inference using a standard data-augmentation approach; the second performs approximate Bayesian inference based on a likelihood approximation derived from branching processes. These methods are compared for computational efficiency and posteriors from each are compared. The branching process is shown to be an excellent approximation and remains computationally efficient as the amount of data is increased.
The premature fusion of cranial bones, craniosynostosis, affects the correct development of the skull producing morphological malformations in newborns. To assess the susceptibility of each craniofacial articulation to close prematurely, we used a network model of the skull to quantify the link reliability (an index based on stochastic block modeling and Bayesian inference) of each articulation. We show that, of the 93 human skull articulations at birth, the few articulations that are associated with nonsyndromic craniosynostosis conditions have statistically significant lower reliability scores than the others. In a similar way, articulations that close during the normal postnatal development of the skull have also lower reliability scores than those articulations that persist through adult live. These results indicate a relationship between the architecture of the skull network and the specific articulations that close during normal development and in pathological conditions. Our findings suggest that the topological arrangement of skull bones might act as an epigenetic factor, predisposing some articulations to closure, both in normal and pathological development, and also affecting the long-term evolution of the skull.
Background: Species abundance distributions in chemical reaction network models cannot usually be computed analytically. Instead, stochas- tic simulation algorithms allow sample from the the system configuration. Although many algorithms have been described, no fast implementation has been provided for {\tau}-leaping which i) is Matlab-compatible, ii) adap- tively alternates between SSA, implicit and explicit {\tau}-leaping, and iii) provides summary statistics necessary for Bayesian inference. Results: We provide a Matlab-compatible implementation of the adap- tive explicit-implicit {\tau}-leaping algorithm to address the above-mentioned deficits. matLeap provides equal or substantially faster results compared to two widely used simulation packages while maintaining accuracy. Lastly, matLeap yields summary statistics of the stochastic process unavailable with other methods, which are indispensable for Bayesian inference. Conclusions: matLeap addresses shortcomings in existing Matlab-compatible stochastic simulation software, providing significant speedups and sum- mary statistics that are especially useful for researchers utilizing particle- filter based methods for Bayesian inference. Code is available for download at https://github.com/claassengroup/matLeap. Contact: [email omitted]