The Neural Networks for Partial Differential Equations (NN4PDEs) approach is used to determine the parameters of a simple land-surface model using PyTorch's backpropagation engine. In order to test the inverse model, a synthetic dataset is created by running the model in forward mode with known parameter values to create soil temperature time series that can be used as observations for the inverse model. We show that it is not possible to obtain a reliable parameter estimation using a single observed soil temperature time series. Using measurements at two depths, reliable parameter estimates can be obtained although it is not possible to differentiate between latent and sensible heat fluxes. We apply the inverse model to urban flux tower data in Phoenix, United States, and show that the thermal conductivity, volumetric heat capacity, and the combined sensible-latent heat transfer coefficient can be reliably estimated using an observed value for the effective surface albedo. The resulting model accurately predicts the outgoing longwave radiation, conductive soil fluxes and the combined sensible-latent heat fluxes.
Normalizing flows (NFs) provide uncorrelated samples from complex distributions, making them an appealing tool for parameter estimation. However, the practical utility of NFs remains limited by their tendency to collapse to a single mode of a multimodal distribution. In this study, we show that annealing with an adaptive schedule based on the effective sample size (ESS) can mitigate mode collapse. We demonstrate that our approach can converge the marginal likelihood for a biochemical oscillator model fit to time-series data in ten-fold less computation time than a widely used ensemble Markov chain Monte Carlo (MCMC) method. We show that the ESS can also be used to reduce variance by pruning the samples. We expect these developments to be of general use for sampling with NFs and discuss potential opportunities for further improvements.
Alzheimer's disease (AD) is a complex neurodegenerative disorder characterized by the accumulation of amyloid-beta (A$\beta$) and phosphorylated tau (p-tau) proteins, leading to cognitive decline measured by the Alzheimer's Disease Assessment Scale (ADAS) score. In this study, we develop and analyze a system of ordinary differential equation models to describe the interactions between A$\beta$, p-tau, and ADAS score, providing a mechanistic understanding of disease progression. To ensure accurate model calibration, we employ Bayesian inference and Physics-Informed Neural Networks (PINNs) for parameter estimation based on Alzheimer's Disease Neuroimaging Initiative data. The data-driven Bayesian approach enables uncertainty quantification, improving confidence in model predictions, while the PINN framework leverages neural networks to capture complex dynamics directly from data. Furthermore, we implement an optimal control strategy to assess the efficacy of an anti-tau therapeutic intervention aimed at reducing p-tau levels and mitigating cognitive decline. Our data-driven solutions indicate that while optimal drug administration effectively decreases p-tau concentration, its impact on cognitive decline, as reflected in the ADAS score, remains limited. These findings suggest that targeting p-tau alone may not be sufficient for significant cognitive improvement, highlighting the need for multi-target therapeutic strategies. The integration of mechanistic modelling, advanced parameter estimation, and control-based therapeutic optimization provides a comprehensive framework for improving treatment strategies for AD.
Accurate estimation of state of health (SOH) is critical for battery applications. Current model-based SOH estimation methods typically rely on low C-rate constant current tests to extract health parameters like solid phase volume fraction and lithium-ion stoichiometry, which are often impractical in real-world scenarios due to time and operational constraints. Additionally, these methods are susceptible to modeling uncertainties that can significantly degrade the estimation accuracy, especially when jointly estimating multiple parameters. In this paper, we present a novel reference voltage-based method for robust battery SOH estimation. This method utilizes the voltage response of a battery under a predefined current excitation at the beginning of life (BOL) as a reference to compensate for modeling uncertainty. As the battery degrades, the same excitation is applied to generate the voltage response, which is compared with the BOL trajectory to estimate the key health parameters accurately. The current excitation is optimally designed using the Particle Swarm Optimization algorithm to maximize the information content of the target parameters. Simulation results demonstrate that our proposed method significantly improves parameter estimation accuracy under different degradation levels, compared to conventional methods relying only on direct voltage measurements. Furthermore, our method jointly estimates four key SOH parameters in only 10 minutes, making it practical for real-world battery health diagnostics, e.g., fast testing to enable battery repurposing.
Agent-based modeling (ABM) is a powerful computational approach for studying complex biological and biomedical systems, yet its widespread use remains limited by significant computational demands. As models become increasingly sophisticated, the number of parameters and interactions rises rapidly, exacerbating the so-called curse of dimensionality and making comprehensive parameter exploration and uncertainty analyses computationally prohibitive. Surrogate modeling provides a promising solution by approximating ABM behavior through computationally efficient alternatives, greatly reducing the runtime needed for parameter estimation, sensitivity analysis, and uncertainty quantification. In this review, we examine traditional approaches for performing these tasks directly within ABMs -- providing a baseline for comparison -- and then synthesize recent developments in surrogate-assisted methodologies for biological and biomedical applications. We cover statistical, mechanistic, and machine-learning-based approaches, emphasizing emerging hybrid strategies that integrate mechanistic insights with machine learning to balance interpretability and scalability. Finally, we discuss current challenges and outline directions for future research, including the development of standardized benchmarks to enhance methodological rigor and facilitate the broad adoption of surrogate-assisted ABMs in biology and medicine.
Integrated sensing and communication (ISAC) has emerged as a key enabler for next-generation wireless networks, supporting advanced applications such as high-precision localization and environment reconstruction. Cooperative ISAC (CoISAC) further enhances these capabilities by enabling multiple base stations (BSs) to jointly optimize communication and sensing performance through coordination. However, CoISAC beamforming design faces significant challenges due to system heterogeneity, large-scale problem complexity, and sensitivity to parameter estimation errors. Traditional deep learning-based techniques fail to exploit the unique structural characteristics of CoISAC systems, thereby limiting their ability to enhance system performance. To address these challenges, we propose a Link-Heterogeneous Graph Neural Network (LHGNN) for joint beamforming in CoISAC systems. Unlike conventional approaches, LHGNN models communication and sensing links as heterogeneous nodes and their interactions as edges, enabling the capture of the heterogeneous nature and intricate interactions of CoISAC systems. Furthermore, a graph attention mechanism is incorporated to dynamically adjust node and link importance, improving robustness to channel and position estimation errors. Numerical results demonstrate that the proposed attention-enhanced LHGNN achieves superior communication rates while maintaining sensing accuracy under power constraints. The proposed method also exhibits strong robustness to communication channel and position estimation error.
This paper provides a practical introduction to Double/Debiased Machine Learning (DML). DML provides a general approach to performing inference about a target parameter in the presence of nuisance parameters. The aim of DML is to reduce the impact of nuisance parameter estimation on estimators of the parameter of interest. We describe DML and its two essential components: Neyman orthogonality and cross-fitting. We highlight that DML reduces functional form dependence and accommodates the use of complex data types, such as text data. We illustrate its application through three empirical examples that demonstrate DML's applicability in cross-sectional and panel settings.
Orthogonal time-frequency space (OTFS) is a potential waveform for integrated sensing and communications (ISAC) systems because it can manage communication and sensing metrics in one unified domain, and has better performance in high mobility scenarios. In practice, a target might come from far distance or with ultra-high speed. However, the max unambiguous range and max tolerable velocity of OTFS-ISAC system is limited by the unambiguous round-trip delay and Doppler shift, which are related to OTFS frame, i.e., time slots and subcarrier spacing, respectively. To enlarge the sensing range, a novel OTFS cross-frame ranging and velocity estimation model as well as its corresponding method based on the Chinese remainder theorem (CRT) are proposed in this paper. By designing co-prime numbers of subcarriers and time slots in different subframes, the difference in the responses of the subframes for a target can be used to estimate the distance and velocity of an out-of-range target. Several frame structures are further designed for specific sensing scenarios, such as target with ultra-high speed or at far distance. Simulation results show that the proposed method can achieve significantly better performance in NMSE compared with the classic sensing methods under the condition of same time and frequency resources.
Parameter estimation for differential equations from measured data is an inverse problem prevalent across quantitative sciences. Physics-Informed Neural Networks (PINNs) have emerged as effective tools for solving such problems, especially with sparse measurements and incomplete system information. However, PINNs face convergence issues, stability problems, overfitting, and complex loss function design. Here we introduce PINNverse, a training paradigm that addresses these limitations by reformulating the learning process as a constrained differential optimization problem. This approach achieves a dynamic balance between data loss and differential equation residual loss during training while preventing overfitting. PINNverse combines the advantages of PINNs with the Modified Differential Method of Multipliers to enable convergence on any point on the Pareto front. We demonstrate robust and accurate parameter estimation from noisy data in four classical ODE and PDE models from physics and biology. Our method enables accurate parameter inference also when the forward problem is expensive to solve.
Hybrid wind-wave energy system, integrating floating offshore wind turbine and wave energy converters, has received much attention in recent years due to its potential benefit in increasing the power harvest density and reducing the levelized cost of electricity. Apart from the design complexities of the hybrid wind-wave energy systems, their energy conversion efficiency, power output smoothness and their safe operations introduce new challenges for their control system designs. Recent studies show that advanced model-based control strategies have the great potential to significantly improve their overall control performance. However the performance of these advanced control strategies rely on the computationally efficient control-oriented models with sufficient fidelity, which are normally difficult to derive due to the complexity of the hydro-, aero-dynamic effects and the couplings.In most available results, the hybrid wind-wave energy system models are established by using the Boundary Element Method, devoting to understanding the hydrodynamic responses and performance analysis. However, such models are complex and involved relatively heavy computational burden, which cannot be directly used for the advanced model-based control methods that are essential for improving power capture efficiency from implementing in practice. To overcome this issue, this paper proposes a control-oriented model of the hybrid windwave energy system with six degrees of freedom. First, ...
Lithium-ion batteries are widely used in transportation, energy storage, and consumer electronics, driving the need for reliable battery management systems (BMS) for state estimation and control. The Single Particle Model (SPM) balances computational efficiency and accuracy but faces challenges in parameter estimation due to numerous parameters. Current SPM models using parabolic approximation introduce intermediate variables and hard to do parameter grouping. This study presents a control-oriented SPM reformulation that employs parameter grouping and parabolic approximation to simplify model parameters while using average and surface lithium-ion concentrations as model output. By parameter grouping, the original 17 parameters were reduced to 9 grouped parameters. The reformulated model achieves a reduced-order ordinary differential equation form while maintaining mathematical accuracy equivalent to the pre-grouped discretized SPM. Through Sobol sensitivity analysis under various current profiles, the grouped parameters were reduced from 9 to 6 highly sensitive parameters. Results demonstrate that estimating these 6 parameters achieves comparable practical accuracy to estimating all 9 parameters, with faster convergence. This control-oriented SPM enhances BMS applications by facilitating state estimation and control while reducing parameter estimation requirements.
Score estimation is the backbone of score-based generative models (SGMs), especially denoising diffusion probabilistic models (DDPMs). A key result in this area shows that with accurate score estimates, SGMs can efficiently generate samples from any realistic data distribution (Chen et al., ICLR'23; Lee et al., ALT'23). This distribution learning result, where the learned distribution is implicitly that of the sampler's output, does not explain how score estimation relates to classical tasks of parameter and density estimation. This paper introduces a framework that reduces score estimation to these two tasks, with various implications for statistical and computational learning theory: Parameter Estimation: Koehler et al. (ICLR'23) demonstrate that a score-matching variant is statistically inefficient for the parametric estimation of multimodal densities common in practice. In contrast, we show that under mild conditions, denoising score-matching in DDPMs is asymptotically efficient. Density Estimation: By linking generation to score estimation, we lift existing score estimation guarantees to $(\epsilon,\delta)$-PAC density estimation, i.e., a function approximating the target log-density within $\epsilon$ on all but a $\delta$-fraction of the space. We provide (i) minimax rates for density estimation over H\"older classes and (ii) a quasi-polynomial PAC density estimation algorithm for the classical Gaussian location mixture model, building on and addressing an open problem from Gatmiry et al. (arXiv'24). Lower Bounds for Score Estimation: Our framework offers the first principled method to prove computational lower bounds for score estimation across general distributions. As an application, we establish cryptographic lower bounds for score estimation in general Gaussian mixture models, conceptually recovering Song's (NeurIPS'24) result and advancing his key open problem.
Infrared unmanned aerial vehicle (UAV) images captured using thermal detectors are often affected by temperature dependent low-frequency nonuniformity, which significantly reduces the contrast of the images. Detecting UAV targets under nonuniform conditions is crucial in UAV surveillance applications. Existing methods typically treat infrared nonuniformity correction (NUC) as a preprocessing step for detection, which leads to suboptimal performance. Balancing the two tasks while enhancing detection beneficial information remains challenging. In this paper, we present a detection-friendly union framework, termed UniCD, that simultaneously addresses both infrared NUC and UAV target detection tasks in an end-to-end manner. We first model NUC as a small number of parameter estimation problem jointly driven by priors and data to generate detection-conducive images. Then, we incorporate a new auxiliary loss with target mask supervision into the backbone of the infrared UAV target detection network to strengthen target features while suppressing the background. To better balance correction and detection, we introduce a detection-guided self-supervised loss to reduce feature discrepancies between the two tasks, thereby enhancing detection robustness to varying nonuniformity levels. Additionally, we construct a new benchmark composed of 50,000 infrared images in various nonuniformity types, multi-scale UAV targets and rich backgrounds with target annotations, called IRBFD. Extensive experiments on IRBFD demonstrate that our UniCD is a robust union framework for NUC and UAV target detection while achieving real-time processing capabilities. Dataset can be available at https://github.com/IVPLaboratory/UniCD.
The increasing complexity of modern neural network architectures demands fast and memory-efficient implementations to mitigate computational bottlenecks. In this work, we evaluate the recently proposed BitNet architecture in HEP applications, assessing its performance in classification, regression, and generative modeling tasks. Specifically, we investigate its suitability for quark-gluon discrimination, SMEFT parameter estimation, and detector simulation, comparing its efficiency and accuracy to state-of-the-art methods. Our results show that while BitNet consistently performs competitively in classification tasks, its performance in regression and generation varies with the size and type of the network, highlighting key limitations and potential areas for improvement.
This paper investigates online identification and prediction for nonlinear stochastic dynamical systems. In contrast to offline learning methods, we develop online algorithms that learn unknown parameters from a single trajectory. A key challenge in this setting is handling the non-independent data generated by the closed-loop system. Existing theoretical guarantees for such systems are mostly restricted to the assumption that inputs are independently and identically distributed (i.i.d.), or that the closed-loop data satisfy a persistent excitation (PE) condition. However, these assumptions are often violated in applications such as adaptive feedback control. In this paper, we propose an online projected Newton-type algorithm for parameter estimation in nonlinear stochastic dynamical systems, and develop an online predictor for system outputs based on online parameter estimates. By using both the stochastic Lyapunov function and martingale estimation methods, we demonstrate that the average regret converges to zero without requiring traditional persistent excitation (PE) conditions. Furthermore, we establish a novel excitation condition that ensures global convergence of the online parameter estimates. The proposed excitation condition is applicable to a broader class of system trajectories, including those violating the PE condition.
We develop and apply a learning framework for parameter estimation in initial value problems that are assessed only indirectly via aggregate data such as sample means and/or standard deviations. Our comprehensive framework follows Bayesian principles and consists of specialized Markov chain Monte Carlo computational schemes that rely on modified Hamiltonian Monte Carlo to align with constraints induced by summary statistics and a novel elliptical slice sampler adapted to the parameters of biological models. We benchmark our methods with synthetic data on microbial growth in batch culture and test them with real growth curve data from laboratory replication experiments on $\textit{Prochlorococcus}$ microbes. The results indicate that our learning framework can utilize experimental or historical data and lead to robust parameter estimation and data assimilation in ODE models that outperform least-squares fitting.
We present a multi-fidelity method for uncertainty quantification of parameter estimates in complex systems, leveraging generative models trained to sample the target conditional distribution. In the Bayesian inference setting, traditional parameter estimation methods rely on repeated simulations of potentially expensive forward models to determine the posterior distribution of the parameter values, which may result in computationally intractable workflows. Furthermore, methods such as Markov Chain Monte Carlo (MCMC) necessitate rerunning the entire algorithm for each new data observation, further increasing the computational burden. Hence, we propose a novel method for efficiently obtaining posterior distributions of parameter estimates for high-fidelity models given data observations of interest. The method first constructs a low-fidelity, conditional generative model capable of amortized Bayesian inference and hence rapid posterior density approximation over a wide-range of data observations. When higher accuracy is needed for a specific data observation, the method employs adaptive refinement of the density approximation. It uses outputs from the low-fidelity generative model to refine the parameter sampling space, ensuring efficient use of the computationally expensive high-fidelity solver. Subsequently, a high-fidelity, unconditional generative model is trained to achieve greater accuracy in the target posterior distribution. Both low- and high- fidelity generative models enable efficient sampling from the target posterior and do not require repeated simulation of the high-fidelity forward model. We demonstrate the effectiveness of the proposed method on several numerical examples, including cases with multi-modal densities, as well as an application in plasma physics for a runaway electron simulation model.
We present a methodology for causal risk analysis in a network. Causal dependence is formulated by a max-linear structural equation model, which expresses each node variable as a max-linear function of its parental node variables in a directed acyclic graph and some exogenous innovation. We determine directed~paths~responsible~for extreme risk propagation in the network. We give algorithms for structure learning and parameter estimation and apply them to a network of financial data.
Despite remarkable advancements in video depth estimation, existing methods exhibit inherent limitations in achieving geometric fidelity through the affine-invariant predictions, limiting their applicability in reconstruction and other metrically grounded downstream tasks. We propose GeometryCrafter, a novel framework that recovers high-fidelity point map sequences with temporal coherence from open-world videos, enabling accurate 3D/4D reconstruction, camera parameter estimation, and other depth-based applications. At the core of our approach lies a point map Variational Autoencoder (VAE) that learns a latent space agnostic to video latent distributions for effective point map encoding and decoding. Leveraging the VAE, we train a video diffusion model to model the distribution of point map sequences conditioned on the input videos. Extensive evaluations on diverse datasets demonstrate that GeometryCrafter achieves state-of-the-art 3D accuracy, temporal consistency, and generalization capability.
In this study, we present a stochastic simulation model designed to explicitly incorporate cell cycle length, overcoming limitations associated with classical compartmental models. Our approach employs a delay mechanism to represent the cell cycle, allowing the use of arbitrary distributions for cell cycle lengths. We demonstrate the feasibility of our model by fitting it to experimental data from melanoma cell lines previously studied by Vittadello et al. Notably, our model successfully replicates experimentally observed synchronization phenomena that multi-stage models could not adequately explain. By using a gamma distribution to model cell cycle lengths, we achieved excellent agreement between our simulations and empirical data, while significantly reducing computational complexity and parameter estimation challenges inherent in multi-stage approaches. Our results highlight the importance of explicitly incorporating cell cycle lengths in modeling cell populations, with potential implications for optimizing cell cycle-dependent tumor therapies.