Data assimilation (DA) combines partial observations with a dynamical model to improve state estimation. Filter-based DA uses only past and present data and is the prerequisite for real-time forecasts. Smoother-based DA exploits both past and future observations. It aims to fill in missing data, provide more accurate estimations, and develop high-quality datasets. However, the standard smoothing procedure requires using all historical state estimations, which is storage-demanding, especially for high-dimensional systems. This paper develops an adaptive-lag online smoother for a large class of complex dynamical systems with strong nonlinear and non-Gaussian features, which has important applications to many real-world problems. The adaptive lag allows the DA to utilize only observations within a nearby window, significantly reducing computational storage. Online lag adjustment is essential for tackling turbulent systems, where temporal autocorrelation varies significantly over time due to intermittency, extreme events, and nonlinearity. Based on the uncertainty reduction in the estimated state, an information criterion is developed to systematically determine the adaptive lag. Notably, the mathematical structure of these systems facilitates the use of closed analytic formulae to calculate the online smoother and the adaptive lag, avoiding empirical tunings as in ensemble-based DA methods. The adaptive online smoother is applied to studying three important scientific problems. First, it helps detect online causal relationships between state variables. Second, its advantage of computational storage is illustrated via Lagrangian DA, a high-dimensional nonlinear problem. Finally, the adaptive smoother advances online parameter estimation with partial observations, emphasizing the role of the observed extreme events in accelerating convergence.
Gaussian processes (GPs) are a ubiquitous tool for geostatistical modeling with high levels of flexibility and interpretability, and the ability to make predictions at unseen spatial locations through a process called Kriging. Estimation of Kriging weights relies on the inversion of the process' covariance matrix, creating a computational bottleneck for large spatial datasets. In this paper, we propose an Amortized Bayesian Local Interpolation NetworK (A-BLINK) for fast covariance parameter estimation, which uses two pre-trained deep neural networks to learn a mapping from spatial location coordinates and covariance function parameters to Kriging weights and the spatial variance, respectively. The fast prediction time of these networks allows us to bypass the matrix inversion step, creating large computational speedups over competing methods in both frequentist and Bayesian settings, and also provides full posterior inference and predictions using Markov chain Monte Carlo sampling methods. We show significant increases in computational efficiency over comparable scalable GP methodology in an extensive simulation study with lower parameter estimation error. The efficacy of our approach is also demonstrated using a temperature dataset of US climate normals for 1991--2020 based on over 7,000 weather stations.
Neurite Orientation Dispersion and Density Imaging (NODDI) microstructure estimation from diffusion magnetic resonance imaging (dMRI) is of great significance for the discovery and treatment of various neurological diseases. Current deep learning-based methods accelerate the speed of NODDI parameter estimation and improve the accuracy. However, most methods require the number and coordinates of gradient directions during testing and training to remain strictly consistent, significantly limiting the generalization and robustness of these models in NODDI parameter estimation. In this paper, we propose a q-space sampling augmentation-based continuous representation learning framework (SamRobNODDI) to achieve robust and generalized NODDI. Specifically, a continuous representation learning method based on q-space sampling augmentation is introduced to fully explore the information between different gradient directions in q-space. Furthermore, we design a sampling consistency loss to constrain the outputs of different sampling schemes, ensuring that the outputs remain as consistent as possible, thereby further enhancing performance and robustness to varying q-space sampling schemes. SamRobNODDI is also a flexible framework that can be applied to different backbone networks. To validate the effectiveness of the proposed method, we compared it with 7 state-of-the-art methods across 18 different q-space sampling schemes, demonstrating that the proposed SamRobNODDI has better performance, robustness, generalization, and flexibility.
We propose a novel coded integrated passive sensing and communication (CIPSAC) system with orthogonal frequency division multiplexing (OFDM), where a multi-antenna base station (BS) passively senses the parameters of the targets and decodes the information bit sequences transmitted by a user. The transmitted signal is comprised of pilot and data OFDM symbols where the data symbols adopt cyclic redundancy check (CRC)-assisted channel codes to facilitate both the decoding and sensing procedures. In the proposed scheme, CRC not only enhances the reliability of communication but also provides guidance to the parameter sensing procedure at the BS. In particular, a novel iterative parameter sensing and channel decoding (IPSCD) algorithm is proposed, where the correctly decoded codewords that pass CRC are utilized for sensing to improve the parameter estimation accuracy, and in return, more accurate parameter estimates lead to a larger number of correctly decoded data symbols. Conventional sensing algorithms rely only on the received pilot signals, while we utilize both the data and pilot signals for sensing. We provide a detailed analysis of the optimal strategy, in which the wrongly decoded data packets are replaced by zero codewords. To further improve the performance, we introduce learning-based near-orthogonal superposition (NOS) codes, which exhibit superior error correction capability especially in the short block length regime. NOS codes are trained using a weighted loss function, where a hyper parameter is introduced to balance the sensing and the communication losses. Simulation results show the effectiveness of the proposed CIPSAC system and the IPSCD algorithm, where both the sensing and decoding performances are significantly improved with a few iterations. We also carry out extensive ablation studies for a comprehensive understanding of the proposed scheme.
Background: Mathematical models based on ordinary differential equations (ODEs) are essential tools across various scientific disciplines, including biology, ecology, and healthcare informatics. They are used to simulate complex dynamic systems and inform decision-making. In this paper, we introduce BayesianFitForecast, an R toolbox specifically developed to streamline Bayesian parameter estimation and forecasting in ODE models, making it particularly relevant to health informatics and public health decision-making. The toolbox is available at https://github.com/gchowell/BayesianFitForecast/. Results: This toolbox enables automatic generation of Stan files, allowing users to configure models, define priors, and analyze results with minimal programming expertise. To demonstrate the versatility and robustness of BayesianFitForecast, we apply it to the analysis of the 1918 influenza pandemic in San Francisco, comparing Poisson and negative binomial error structures within the SEIR model. We also test it by fitting multiple time series of state variables using simulated data. BayesianFitForecast provides robust tools for evaluating model performance, including convergence diagnostics, posterior distributions, credible intervals, and performance metrics. Conclusion: By improving the accessibility of advanced Bayesian methods, this toolbox significantly broadens the application of Bayesian inference methods to dynamical systems critical for healthcare and epidemiological forecasting. A tutorial video demonstrating the toolbox's functionality is available at https://youtu.be/jnxMjz3V3n8.
Large Language Models (LLMs) are highly resource-intensive to fine-tune due to their enormous size. While low-rank adaptation is a prominent parameter-efficient fine-tuning approach, it suffers from sensitivity to hyperparameter choices, leading to instability in model performance on fine-tuning downstream tasks. This paper highlights the importance of effective parameterization in low-rank fine-tuning to reduce estimator variance and enhance the stability of final model outputs. We propose MonteCLoRA, an efficient fine-tuning technique, employing Monte Carlo estimation to learn an unbiased posterior estimation of low-rank parameters with low expected variance, which stabilizes fine-tuned LLMs with only O(1) additional parameters. MonteCLoRA shows significant improvements in accuracy and robustness, achieving up to 3.8% higher accuracy and 8.6% greater robustness than existing efficient fine-tuning methods on natural language understanding tasks with pre-trained RoBERTa-base. Furthermore, in generative tasks with pre-trained LLaMA-1-7B, MonteCLoRA demonstrates robust zero-shot performance with 50% lower variance than the contemporary efficient fine-tuning methods. The theoretical and empirical results presented in the paper underscore how parameterization and hyperpriors balance exploration-exploitation in the low-rank parametric space, therefore leading to more optimal and robust parameter estimation during efficient fine-tuning.
The Fisher Information Metric (FIM) and the associated Cram\'er-Rao Bound (CRB) are fundamental tools in statistical signal processing, which inform the efficient design of experiments and algorithms for estimating the underlying parameters. In this article, we investigate these concepts for the case where the parameters lie on a homogeneous space. Unlike the existing Fisher-Rao theory for general Riemannian manifolds, our focus is to leverage the group-theoretic structure of homogeneous spaces, which is often much easier to work with than their Riemannian structure. The FIM is characterized by identifying the homogeneous space with a coset space, the group-theoretic CRB and its corollaries are presented, and its relationship to the Riemannian CRB is clarified. The application of our theory is illustrated using two examples from engineering: (i) estimation of the pose of a robot and (ii) sensor network localization. In particular, these examples demonstrate that homogeneous spaces provide a natural framework for studying statistical models that are invariant with respect to a group of symmetries.
This paper presents a novel parametric scattering model (PSM) for sensing extended targets in integrated sensing and communication (ISAC) systems. The PSM addresses the limitations of traditional models by efficiently capturing the target's angular characteristics through a compact set of key parameters, including the central angle and angular spread, enabling efficient optimization. Based on the PSM, we first derive the Cramer-Rao Bound (CRB) for parameter estimation and then propose a beamforming design algorithm to minimize the CRB while meeting both communication signal-to-interference-plus-noise ratio (SINR) and power constraints. By integrating the PSM into the beamforming optimization process, the proposed framework achieves superior CRB performance while balancing the tradeoff between sensing accuracy and communication quality. Simulation results demonstrate that the PSM-based approach consistently outperforms traditional unstructured and discrete scattering models, particularly in resource-limited scenarios, highlighting its practical applicability and scalability.
Integrated Sensing and Communication (ISAC) is a technology paradigm that combines sensing capabilities with communication functionalities in a single device or system. In vehicle-to-everything (V2X) sidelink, ISAC can provide enhanced safety by allowing vehicles to not only communicate with one another but also sense the surrounding environment by using sidelink signals. In ISAC-capable V2X sidelink, the random resource allocation results in an unstructured and sparse distribution of time and frequency resources in the received orthogonal frequency division multiplexing (OFDM) grid, leading to degraded radar detection performance when processed using the conventional 2D-FFT method. To address this challenge, this paper proposes a high-resolution off-grid radar target detection algorithm irrespective of the OFDM grid structure. The proposed method utilizes the Newtonized orthogonal matching pursuit (NOMP) algorithm to effectively detect weak targets masked by the sidelobes of stronger ones and accurately estimates off-grid range and velocity parameters with minimal resources through Newton refinements. Simulation results demonstrate the superior performance of the proposed NOMP-based target detection algorithm compared to existing compressed sensing (CS) methods in terms of detection probability, resolution, and accuracy. Additionally, experimental validation is performed using a bi-static radar setup in a semi-anechoic chamber. The measurement results validate the simulation findings, showing that the proposed algorithm significantly enhances target detection and parameter estimation accuracy in realistic scenarios.
This study proposes a modularized deep learning-based loading protocol for optimal parameter estimation of Bouc-Wen (BW) class models. The protocol consists of two key components: optimal loading history construction and CNN-based rapid parameter estimation. Each component is decomposed into independent sub-modules tailored to distinct hysteretic behaviors-basic hysteresis, structural degradation, and pinching effect-making the protocol adaptable to diverse hysteresis models. Three independent CNN architectures are developed to capture the path-dependent nature of these hysteretic behaviors. By training these CNN architectures on diverse loading histories, minimal loading sequences, termed \textit{loading history modules}, are identified and then combined to construct an optimal loading history. The three CNN models, trained on the respective loading history modules, serve as rapid parameter estimators. Numerical evaluation of the protocol, including nonlinear time history analysis of a 3-story steel moment frame and fragility curve construction for a 3-story reinforced concrete frame, demonstrates that the proposed protocol significantly reduces total analysis time while maintaining or improving estimation accuracy. The proposed protocol can be extended to other hysteresis models, suggesting a systematic approach for identifying general hysteresis models.
This material introduces the D-Subspace algorithm derived on the basis of the centralized algorithm [1], which originally addresses parameter estimation problems under a subspace constraint.
The task of Novel View Acoustic Synthesis (NVAS) - generating Room Impulse Responses (RIRs) for unseen source and receiver positions in a scene - has recently gained traction, especially given its relevance to Augmented Reality (AR) and Virtual Reality (VR) development. However, many of these efforts suffer from similar limitations: they infer RIRs in the time domain, which prove challenging to optimize; they focus on scenes with simple, single-room geometries; they infer only single-channel, directionally-independent acoustic characteristics; and they require inputs, such as 3D geometry meshes with material properties, that may be impractical to obtain for on-device applications. On the other hand, research suggests that sample-wise accuracy of RIRs is not required for perceptual plausibility in AR and VR. Standard acoustic parameters like Clarity Index (C50) or Reverberation Time (T60) have been shown to capably describe pertinent characteristics of the RIRs, especially late reverberation. To address these gaps, this paper introduces a new task centered on estimating spatially distributed acoustic parameters that can be then used to condition a simple reverberator for arbitrary source and receiver positions. The approach is modelled as an image-to-image translation task, which translates 2D floormaps of a scene into 2D heatmaps of acoustic parameters. We introduce a new, large-scale dataset of 1000 scenes consisting of complex, multi-room apartment conditions, and show that our method outperforms statistical baselines significantly. Moreover, we show that the method also works for directionally-dependent (i.e. beamformed) parameter prediction. Finally, the proposed method operates on very limited information, requiring only a broad outline of the scene and a single RIR at inference time.
Thermal states play a fundamental role in various areas of physics, and they are becoming increasingly important in quantum information science, with applications related to semi-definite programming, quantum Boltzmann machine learning, Hamiltonian learning, and the related task of estimating the parameters of a Hamiltonian. Here we establish formulas underlying the basic geometry of parameterized thermal states, and we delineate quantum algorithms for estimating the values of these formulas. More specifically, we prove formulas for the Fisher--Bures and Kubo--Mori information matrices of parameterized thermal states, and our quantum algorithms for estimating their matrix elements involve a combination of classical sampling, Hamiltonian simulation, and the Hadamard test. These results have applications in developing a natural gradient descent algorithm for quantum Boltzmann machine learning, which takes into account the geometry of thermal states, and in establishing fundamental limitations on the ability to estimate the parameters of a Hamiltonian, when given access to thermal-state samples. For the latter task, and for the special case of estimating a single parameter, we sketch an algorithm that realizes a measurement that is asymptotically optimal for the estimation task. We finally stress that the natural gradient descent algorithm developed here can be used for any machine learning problem that employs the quantum Boltzmann machine ansatz.
In a time-division broadcast positioning system (TDBPS), localizing mobile targets using classical time difference of arrival (TDOA) methods poses significant challenges. Concurrent TDOA measurements are infeasible because targets receive signals from different anchors and extract their transmission times at different reception times, as well as at varying positions. Traditional TDOA estimation schemes implicitly assume that the target remains stationary during the measurement period, which is impractical for mobile targets exhibiting high dynamics. Existing methods for mobile target localization are mostly specialized and rely on motion modeling and do not rely on the concurrent TDOA measurements. This issue limits their direct use of the well-established classical TDOA-based localization methods and complicating the entire localization process. In this paper, to obtain concurrent TDOA estimates at any instant out of the sequential measurements for direct use of existing TDOA-based localization methods, we propose a novel TDOA estimation method, termed parameterized TDOA (P-TDOA). By approximating the time-varying TDOA as a polynomial function over a short period, we transform the TDOA estimation problem into a model parameter estimation problem and derive the desired TDOA estimates thereafter. Theoretical analysis shows that, under certain conditions, the proposed P-TDOA method closely approaches the Cramer-Rao Lower Bound (CRLB) for TDOA estimation in concurrent measurement scenarios, despite measurements being obtained sequentially. Extensive numerical simulations validate our theoretical analysis and demonstrate the effectiveness of the proposed method, highlighting substantial improvements over existing approaches across various scenarios.
The paper proposes and implements a methodology to fit a seven-parameter Generalized Tempered Stable (GTS) distribution to financial data. The nonexistence of the mathematical expression of the GTS probability density function makes the maximum likelihood estimation (MLE) inadequate for providing parameter estimations. Based on the function characteristic and the fractional Fourier transform (FRFT), we provide a comprehensive approach to circumvent the problem and yield a good parameter estimation of the GTS probability. The methodology was applied to fit two heavily tailed data (Bitcoin and Ethereum returns) and two peaked data (S\&P 500 and SPY ETF returns). For each index, the estimation results show that the six-parameter estimations are statistically significant except for the local parameter, $\mu$. The goodness-of-fit was assessed through Kolmogorov-Smirnov, Anderson-Darling, and Pearson's chi-squared statistics. While the two-parameter geometric Brownian motion (GBM) hypothesis is always rejected, the GTS distribution fits significantly with a very high p-value; and outperforms the Kobol, Carr-Geman-Madan-Yor, and Bilateral Gamma distributions.
The analysis of neural power spectra plays a crucial role in understanding brain function and dysfunction. While recent efforts have led to the development of methods for decomposing spectral data, challenges remain in performing statistical analysis and group-level comparisons. Here, we introduce Bayesian Spectral Decomposition (BSD), a Bayesian framework for analysing neural spectral power. BSD allows for the specification, inversion, comparison, and analysis of parametric models of neural spectra, addressing limitations of existing methods. We first establish the face validity of BSD on simulated data and show how it outperforms an established method (\fooof{}) for peak detection on artificial spectral data. We then demonstrate the efficacy of BSD on a group-level study of EEG spectra in 204 healthy subjects from the LEMON dataset. Our results not only highlight the effectiveness of BSD in model selection and parameter estimation, but also illustrate how BSD enables straightforward group-level regression of the effect of continuous covariates such as age. By using Bayesian inference techniques, BSD provides a robust framework for studying neural spectral data and their relationship to brain function and dysfunction.
We present an accelerated pipeline, based on high-performance computing techniques and normalizing flows, for joint Bayesian parameter estimation and model selection and demonstrate its efficiency in gravitational wave astrophysics. We integrate the Jim inference toolkit, a normalizing flow-enhanced Markov chain Monte Carlo (MCMC) sampler, with the learned harmonic mean estimator. Our Bayesian evidence estimates run on $1$ GPU are consistent with traditional nested sampling techniques run on $16$ CPU cores, while reducing the computation time by factors of $5\times$ and $15\times$ for $4$-dimensional and $11$-dimensional gravitational wave inference problems, respectively. Our code is available in well-tested and thoroughly documented open-source packages, ensuring accessibility and reproducibility for the wider research community.
Sim-to-real transfer remains a significant challenge in robotics due to the discrepancies between simulated and real-world dynamics. Traditional methods like Domain Randomization often fail to capture fine-grained dynamics, limiting their effectiveness for precise control tasks. In this work, we propose a novel approach that dynamically adjusts simulation environment parameters online using in-context learning. By leveraging past interaction histories as context, our method adapts the simulation environment dynamics to real-world dynamics without requiring gradient updates, resulting in faster and more accurate alignment between simulated and real-world performance. We validate our approach across two tasks: object scooping and table air hockey. In the sim-to-sim evaluations, our method significantly outperforms the baselines on environment parameter estimation by 80% and 42% in the object scooping and table air hockey setups, respectively. Furthermore, our method achieves at least 70% success rate in sim-to-real transfer on object scooping across three different objects. By incorporating historical interaction data, our approach delivers efficient and smooth system identification, advancing the deployment of robots in dynamic real-world scenarios. Demos are available on our project page: https://sim2real-capture.github.io/
Identifying dynamical systems characterized by nonlinear parameters presents significant challenges in deriving mathematical models that enhance understanding of physics. Traditional methods, such as Sparse Identification of Nonlinear Dynamics (SINDy) and symbolic regression, can extract governing equations from observational data; however, they also come with distinct advantages and disadvantages. This paper introduces a novel method within the SINDy framework, termed ADAM-SINDy, which synthesizes the strengths of established approaches by employing the ADAM optimization algorithm. This facilitates the simultaneous optimization of nonlinear parameters and coefficients associated with nonlinear candidate functions, enabling precise parameter estimation without requiring prior knowledge of nonlinear characteristics such as trigonometric frequencies, exponential bandwidths, or polynomial exponents, thereby addressing a key limitation of SINDy. Through an integrated global optimization, ADAM-SINDy dynamically adjusts all unknown variables in response to data, resulting in an adaptive identification procedure that reduces the sensitivity to the library of candidate functions. The performance of the ADAM-SINDy methodology is demonstrated across a spectrum of dynamical systems, including benchmark coupled nonlinear ordinary differential equations such as oscillators, chaotic fluid flows, reaction kinetics, pharmacokinetics, as well as nonlinear partial differential equations (wildfire transport). The results demonstrate significant improvements in identifying parameterized dynamical systems and underscore the importance of concurrently optimizing all parameters, particularly those characterized by nonlinear parameters. These findings highlight the potential of ADAM-SINDy to extend the applicability of the SINDy framework in addressing more complex challenges in dynamical system identification.
This paper proposes a computationally tractable algorithm for learning infinite-horizon average-reward linear mixture Markov decision processes (MDPs) under the Bellman optimality condition. Our algorithm for linear mixture MDPs achieves a nearly minimax optimal regret upper bound of $\widetilde{\mathcal{O}}(d\sqrt{\mathrm{sp}(v^*)T})$ over $T$ time steps where $\mathrm{sp}(v^*)$ is the span of the optimal bias function $v^*$ and $d$ is the dimension of the feature mapping. Our algorithm applies the recently developed technique of running value iteration on a discounted-reward MDP approximation with clipping by the span. We prove that the value iteration procedure, even with the clipping operation, converges. Moreover, we show that the associated variance term due to random transitions can be bounded even under clipping. Combined with the weighted ridge regression-based parameter estimation scheme, this leads to the nearly minimax optimal regret guarantee.