Generalized linear mixed models (GLMMs) are widely used for analyzing correlated data, particularly in large-scale biomedical and social science applications. Scalable Bayesian inference for GLMMs is challenging due to an intractable marginal likelihood and a high computational cost incurred by conventional Markov chain Monte Carlo (MCMC) methods. We develop a stochastic gradient MCMC (SGMCMC) algorithm tailored to GLMMs that enables accurate posterior inference in the large-sample regime. Our approach uses Fisher's identity to construct a (biased) Monte Carlo estimator of the gradient of the marginal log-likelihood, making SGMCMC feasible when direct gradient computation is impossible. We analyze the additional variability, introduced by both data subsampling and gradient approximation, to derive a post-hoc covariance correction that yields properly calibrated posterior uncertainty. We show through simulated studies that the proposed method provides accurate posterior means and variances in settings with a large number of groups, outperforming existing approaches, including control variate methods. We further demonstrate the method's practical utility in an analysis of electronic health records data, where accounting for variance inflation materially changes scientific conclusions.
Parameter estimation is a fundamental challenge in machine learning, crucial for tasks such as neural network weight fitting and Bayesian inference. This paper focuses on the complexity of estimating translation $\boldsymbol{\mu} \in \mathbb{R}^l$ and shrinkage $\sigma \in \mathbb{R}_{++}$ parameters for a distribution of the form $\frac{1}{\sigma^l} f_0 \left( \frac{\boldsymbol{x} - \boldsymbol{\mu}}{\sigma} \right)$, where $f_0$ is a known density in $\mathbb{R}^l$ given $n$ samples. We highlight that while the problem is NP-hard for Maximum Likelihood Estimation (MLE), it is possible to obtain $\varepsilon$-approximations for arbitrary $\varepsilon > 0$ within $\text{poly} \left( \frac{1}{\varepsilon} \right)$ time using the Wasserstein distance.
This textbook gives a systematic treatment of statistical machine learning for astronomical research, based on Bayesian inference. It shows how modern data analysis techniques connect to classical statistical methods and follow from familiar statistical principles. The Bayesian perspective, used throughout, gives priority to uncertainty quantification and statistical rigor, which scientific inference in astronomy needs. The textbook progresses from probability theory and Bayesian inference through supervised learning, including linear regression with measurement uncertainties, logistic regression and classification. Unsupervised learning topics cover Principal Component Analysis and clustering methods. We then introduce sampling and Markov Chain Monte Carlo, followed by Gaussian Processes as probabilistic nonparametric methods and neural networks in the broader statistical context. Our theory-focused approach derives each method from first principles with complete mathematical development and then shows its astronomical applications. We put statistical insight first and explain why algorithms work, when they are appropriate, and how they connect to broader statistical principles. From a solid grounding in classical methods and their theory, the treatment leads to modern techniques such as neural networks. With this grounding, readers can apply the methods to astronomical research with proper attention to assumptions, limits and uncertainty propagation, as large astronomical surveys require. The 22 companion tutorials apply the methods to astronomical data in Jupyter notebooks.