Clustering is essential in data analysis and machine learning, but traditional algorithms like $k$-means and Gaussian Mixture Models (GMM) often fail with nonconvex clusters. To address the challenge, we introduce the Flexible Bivariate Beta Mixture Model (FBBMM), which utilizes the flexibility of the bivariate beta distribution to handle diverse and irregular cluster shapes. Using the Expectation Maximization (EM) algorithm and Sequential Least Squares Programming (SLSQP) optimizer for parameter estimation, we validate FBBMM on synthetic and real-world datasets, demonstrating its superior performance in clustering complex data structures, offering a robust solution for big data analytics across various domains. We release the experimental code at https://github.com/yung-peng/MBMM-and-FBBMM.
Cell-free (CF) integrated sensing and communication (ISAC) combines CF architecture with ISAC. CF employs distributed access points, eliminates cell boundaries, and enhances coverage, spectral efficiency, and reliability. ISAC unifies radar sensing and communication, enabling simultaneous data transmission and environmental sensing within shared spectral and hardware resources. CF-ISAC leverages these strengths to improve spectral and energy efficiency while enhancing sensing in wireless networks. As a promising candidate for next-generation wireless systems, CF-ISAC supports robust multi-user communication, distributed multi-static sensing, and seamless resource optimization. However, a comprehensive survey on CF-ISAC has been lacking. This paper fills that gap by first revisiting CF and ISAC principles, covering cooperative transmission, radar cross-section, target parameter estimation, ISAC integration levels, sensing metrics, and applications. It then explores CF-ISAC systems, emphasizing their unique features and the benefits of multi-static sensing. State-of-the-art developments are categorized into performance analysis, resource allocation, security, and user/target-centric designs, offering a thorough literature review and case studies. Finally, the paper identifies key challenges such as synchronization, multi-target detection, interference management, and fronthaul capacity and latency. Emerging trends, including next-generation antenna technologies, network-assisted systems, near-field CF-ISAC, integration with other technologies, and machine learning approaches, are highlighted to outline the future trajectory of CF-ISAC research.
Positioning technology, which aims to determine the geometric information of a device in a global coordinate, is a key component in integrated sensing and communication systems. In addition to traditional active anchor-based positioning systems, reconfigurable intelligent surfaces (RIS) have shown great potential for enhancing system performance. However, their ability to manipulate electromagnetic waves and ease of deployment pose potential risks, as unauthorized RIS may be intentionally introduced to jeopardize the positioning service. Such an unauthorized RIS can cause unexpected interference in the original localization system, distorting the transmitted signals, and leading to degraded positioning accuracy. In this work, we investigate the scenario of RIS-aided positioning in the presence of interference from an unauthorized RIS. Theoretical lower bounds are employed to analyze the impact of unauthorized RIS on channel parameter estimation and positioning accuracy. Several codebook design strategies for unauthorized RIS are evaluated, and various system arrangements are discussed. The simulation results show that an unauthorized RIS path with a high channel gain or a delay similar to that of legitimate RIS paths leads to poor positioning performance. Furthermore, unauthorized RIS generates more effective interference when using directional beamforming codebooks compared to random codebooks.
Building articulated objects is a key challenge in computer vision. Existing methods often fail to effectively integrate information across different object states, limiting the accuracy of part-mesh reconstruction and part dynamics modeling, particularly for complex multi-part articulated objects. We introduce ArtGS, a novel approach that leverages 3D Gaussians as a flexible and efficient representation to address these issues. Our method incorporates canonical Gaussians with coarse-to-fine initialization and updates for aligning articulated part information across different object states, and employs a skinning-inspired part dynamics modeling module to improve both part-mesh reconstruction and articulation learning. Extensive experiments on both synthetic and real-world datasets, including a new benchmark for complex multi-part objects, demonstrate that ArtGS achieves state-of-the-art performance in joint parameter estimation and part mesh reconstruction. Our approach significantly improves reconstruction quality and efficiency, especially for multi-part articulated objects. Additionally, we provide comprehensive analyses of our design choices, validating the effectiveness of each component to highlight potential areas for future improvement. Our work is made publicly available at: https://articulate-gs.github.io.
The paper addresses the problem of parameter estimation (or identification) in dynamical networks composed of an arbitrary number of FitzHugh-Nagumo neuron models with diffusive couplings between each other. It is assumed that only the membrane potential of each model is measured, while the other state variable and all derivatives remain unmeasured. Additionally, potential measurement errors in the membrane potential due to sensor imprecision are considered. To solve this problem, firstly, the original FitzHugh-Nagumo network is transformed into a linear regression model, where the regressors are obtained by applying a filter-differentiator to specific combinations of the measured variables. Secondly, the speed-gradient method is applied to this linear model, leading to the design of an identification algorithm for the FitzHugh-Nagumo neural network. Sufficient conditions for the asymptotic convergence of the parameter estimates to their true values are derived for the proposed algorithm. Parameter estimation for a network of five interconnected neurons is demonstrated through computer simulation. The results confirm that the sufficient conditions are satisfied in the numerical experiments conducted. Furthermore, the algorithm's capabilities for adjusting the identification accuracy and time are investigated. The proposed approach has potential applications in nervous system modeling, particularly in the context of human brain modeling. For instance, EEG signals could serve as the measured variables of the network, enabling the integration of mathematical neural models with empirical data collected by neurophysiologists.
In statistics, generalized linear models (GLMs) are widely used for modeling data and can expressively capture potential nonlinear dependence of the model's outcomes on its covariates. Within the broad family of GLMs, those with binary outcomes, which include logistic and probit regressions, are motivated by common tasks such as binary classification with (possibly) non-separable data. In addition, in modern machine learning and statistics, data is often high-dimensional yet has a low intrinsic dimension, making sparsity constraints in models another reasonable consideration. In this work, we propose to use and analyze an iterative hard thresholding (projected gradient descent on the ReLU loss) algorithm, called binary iterative hard thresholding (BIHT), for parameter estimation in sparse GLMs with binary outcomes. We establish that BIHT is statistically efficient and converges to the correct solution for parameter estimation in a general class of sparse binary GLMs. Unlike many other methods for learning GLMs, including maximum likelihood estimation, generalized approximate message passing, and GLM-tron (Kakade et al. 2011; Bahmani et al. 2016), BIHT does not require knowledge of the GLM's link function, offering flexibility and generality in allowing the algorithm to learn arbitrary binary GLMs. As two applications, logistic and probit regression are additionally studied. In this regard, it is shown that in logistic regression, the algorithm is in fact statistically optimal in the sense that the order-wise sample complexity matches (up to logarithmic factors) the lower bound obtained previously. To the best of our knowledge, this is the first work achieving statistical optimality for logistic regression in all noise regimes with a computationally efficient algorithm. Moreover, for probit regression, our sample complexity is on the same order as that obtained for logistic regression.
The degree-corrected block model (DCBM), latent space model (LSM), and $\beta$-model are all popular network models. We combine their modeling ideas and propose the logit-DCBM as a new model. Similar as the $\beta$-model and LSM, the logit-DCBM contains nonlinear factors, where fitting the parameters is a challenging open problem. We resolve this problem by introducing a cancellation trick. We also propose R-SCORE as a recursive community detection algorithm, where in each iteration, we first use the idea above to update our parameter estimation, and then use the results to remove the nonlinear factors in the logit-DCBM so the renormalized model approximately satisfies a low-rank model, just like the DCBM. Our numerical study suggests that R-SCORE significantly improves over existing spectral approaches in many cases. Also, theoretically, we show that the Hamming error rate of R-SCORE is faster than that of SCORE in a specific sparse region, and is at least as fast outside this region.
Data-driven inverse optimization seeks to estimate unknown parameters in an optimization model from observations of optimization solutions. Many existing methods are ineffective in handling noisy and suboptimal solution observations and also suffer from computational challenges. In this paper, we build a connection between inverse optimization and the Fenchel-Young (FY) loss originally designed for structured prediction, proposing a FY loss approach to data-driven inverse optimization. This new approach is amenable to efficient gradient-based optimization, hence much more efficient than existing methods. We provide theoretical guarantees for the proposed method and use extensive simulation and real-data experiments to demonstrate its significant advantage in parameter estimation accuracy, decision error and computational speed.
Accurate underwater navigation is a challenging task due to the absence of global navigation satellite system signals and the reliance on inertial navigation systems that suffer from drift over time. Doppler velocity logs (DVLs) are typically used to mitigate this drift through velocity measurements, which are commonly estimated using a parameter estimation approach such as least squares (LS). However, LS works under the assumption of ideal conditions and does not account for sensor biases, leading to suboptimal performance. This paper proposes a data-driven alternative based on multi-output Gaussian process regression (MOGPR) to improve DVL velocity estimation. MOGPR provides velocity estimates and associated measurement covariances, enabling an adaptive integration within an error-state Extended Kalman Filter (EKF). We evaluate our proposed approach using real-world AUV data and compare it against LS and a state-of-the-art deep learning model, BeamsNet. Results demonstrate that MOGPR reduces velocity estimation errors by approximately 20% while simultaneously enhancing overall navigation accuracy, particularly in the orientation states. Additionally, the incorporation of uncertainty estimates from MOGPR enables an adaptive EKF framework, improving navigation robustness in dynamic underwater environments.
A near-field motion parameter estimation method is proposed. In contract to far-field sensing systems, the near-field sensing system leverages spherical-wave characteristics to enable full-vector location and velocity estimation. Despite promising advantages, the near-field sensing system faces a significant challenge, where location and velocity parameters are intricately coupled within the signal. To address this challenge, a novel subarray-based variational message passing (VMP) method is proposed for near-field joint location and velocity estimation. First, a factor graph representation is introduced, employing subarray-level directional and Doppler parameters as intermediate variables to decouple the complex location-velocity dependencies. Based on this, the variational Bayesian inference is employed to obtain closed-form posterior distributions of subarray-level parameters. Subsequently, the message passing technique is employed, enabling tractable computation of location and velocity marginal distributions. Two implementation strategies are proposed: 1) System-level fusion that aggregates all subarray posteriors for centralized estimation, or 2) Subarray-level fusion where locally processed estimates from subarrays are fused through Guassian product rule. Cram\'er-Rao bounds for location and velocity estimation are derived, providing theoretical performance limits. Numerical results demonstrate that the proposed VMP method outperforms existing approaches while achieving a magnitude lower complexity. Specifically, the proposed VMP method achieves centimeter-level location accuracy and sub-m/s velocity accuracy. It also demonstrates robust performance for high-mobility targets, making the proposed VMP method suitable for real-time near-field sensing and communication applications.
This paper investigates the use of beyond diagonal reconfigurable intelligent surface (BD-RIS) with $N$ elements to advance integrated sensing and communication (ISAC). We address a key gap in the statistical characterizations of the radar signal-to-noise ratio (SNR) and the communication signal-to-interference-plus-noise ratio (SINR) by deriving tractable closed-form cumulative distribution functions (CDFs) for these metrics. Our approach maximizes the radar SNR by jointly configuring radar beamforming and BD-RIS phase shifts. Subsequently, zero-forcing is adopted to mitigate user interference, enhancing the communication SINR. To meet ISAC outage requirements, we propose an analytically-driven successive non-inversion sampling (SNIS) algorithm for estimating network parameters satisfying network outage constraints. Numerical results illustrate the accuracy of the derived CDFs and demonstrate the effectiveness of the proposed SNIS algorithm.
This work demonstrates that the tools and principles driving the success of large language models (LLMs) can be repurposed to tackle distribution-level tasks, where the goal is to predict properties of the data-generating distribution rather than labels for individual datapoints. These tasks encompass statistical inference problems such as parameter estimation, hypothesis testing, or mutual information estimation. Framing these tasks within traditional machine learning pipelines is challenging, as supervision is typically tied to individual datapoint. We propose meta-statistical learning, a framework inspired by multi-instance learning that reformulates statistical inference tasks as supervised learning problems. In this approach, entire datasets are treated as single inputs to neural networks, which predict distribution-level parameters. Transformer-based architectures, without positional encoding, provide a natural fit due to their permutation-invariance properties. By training on large-scale synthetic datasets, meta-statistical models can leverage the scalability and optimization infrastructure of Transformer-based LLMs. We demonstrate the framework's versatility with applications in hypothesis testing and mutual information estimation, showing strong performance, particularly for small datasets where traditional neural methods struggle.
Advances in deep learning and representation learning have transformed item factor analysis (IFA) in the item response theory (IRT) literature by enabling more efficient and accurate parameter estimation. Variational Autoencoders (VAEs) have been one of the most impactful techniques in modeling high-dimensional latent variables in this context. However, the limited expressiveness of the inference model based on traditional VAEs can still hinder the estimation performance. We introduce Adversarial Variational Bayes (AVB) algorithms as an improvement to VAEs for IFA with improved flexibility and accuracy. By bridging the strengths of VAEs and Generative Adversarial Networks (GANs), AVB incorporates an auxiliary discriminator network to reframe the estimation process as a two-player adversarial game and removes the restrictive assumption of standard normal distributions in the inference model. Theoretically, AVB can achieve similar or higher likelihood compared to VAEs. A further enhanced algorithm, Importance-weighted Adversarial Variational Bayes (IWAVB) is proposed and compared with Importance-weighted Autoencoders (IWAE). In an exploratory analysis of empirical data, IWAVB demonstrated superior expressiveness by achieving a higher likelihood compared to IWAE. In confirmatory analysis with simulated data, IWAVB achieved similar mean-square error results to IWAE while consistently achieving higher likelihoods. When latent variables followed a multimodal distribution, IWAVB outperformed IWAE. With its innovative use of GANs, IWAVB is shown to have the potential to extend IFA to handle large-scale data, facilitating the potential integration of psychometrics and multimodal data analysis.
Uniform and smooth data collection is often infeasible in real-world scenarios. In this paper, we propose an identification framework to effectively handle the so-called non-uniform observations, i.e., data scenarios that include missing measurements, multiple runs, or aggregated observations. The goal is to provide a general approach for accurately recovering the overall dynamics of possibly nonlinear systems, allowing the capture of the system behavior over time from non-uniform observations. The proposed approach exploits prior knowledge by integrating domain-specific, interpretable, physical principles with black-box approximators, proving significant flexibility and adaptability in handling different types of non-uniform measurements, and addressing the limitations of traditional linear and black-box methods. The description of this novel framework is supported by a theoretical study on the effect of non-uniform observations on the accuracy of parameter estimation. Specifically, we demonstrate the existence of upper bounds on the parametric error resulting from missing measurements and aggregated observations. Then, the effectiveness of the approach is demonstrated through two case studies. These include a practical application with missing samples, i.e., the identification of a continuous stirred-tank reactor using real data, and a simulated Lotka-Volterra system under aggregated observations. The results highlight the ability of the framework to robustly estimate the system parameters and to accurately reconstruct the model dynamics despite the availability of non-uniform measurements.
Numerous real-world control problems involve dynamics and objectives affected by unobservable hidden pa- rameters, ranging from autonomous driving to robotic manipu- lation, which cause performance degradation during sim-to-real transfer. To represent these kinds of domains, we adopt hidden- parameter Markov decision processes (HIP-MDPs), which model sequential decision problems where hidden variables parameterize transition and reward functions. Existing ap- proaches, such as domain randomization, domain adaptation, and meta-learning, simply treat the effect of hidden param- eters as additional variance and often struggle to effectively handle HIP-MDP problems, especially when the rewards are parameterized by hidden variables. We introduce Privileged- Dreamer, a model-based reinforcement learning framework that extends the existing model-based approach by incorporating an explicit parameter estimation module. PrivilegedDreamer features its novel dual recurrent architecture that explicitly estimates hidden parameters from limited historical data and enables us to condition the model, actor, and critic networks on these estimated parameters. Our empirical analysis on five diverse HIP-MDP tasks demonstrates that PrivilegedDreamer outperforms state-of-the-art model-based, model-free, and do- main adaptation learning algorithms. Additionally, we conduct ablation studies to justify the inclusion of each component in the proposed architecture.
The Weak-form Estimation of Non-linear Dynamics (WENDy) algorithm is extended to accommodate systems of ordinary differential equations that are nonlinear-in-parameters. The extension rests on derived analytic expressions for a likelihood function, its gradient and its Hessian matrix. WENDy makes use of these to approximate a maximum likelihood estimator based on optimization routines suited for non-convex optimization problems. The resulting parameter estimation algorithm has better accuracy, a substantially larger domain of convergence, and is often orders of magnitude faster than the conventional output error least squares method (based on forward solvers). The algorithm is efficiently implemented in Julia. We demonstrate the algorithm's ability to accommodate the weak form optimization for both additive normal and multiplicative log-normal noise, and present results on a suite of benchmark systems of ordinary differential equations. In order to demonstrate the practical benefits of our approach, we present extensive comparisons between our method and output error methods in terms of accuracy, precision, bias, and coverage.
Background: MRI is crucial for brain imaging but is highly susceptible to motion artifacts due to long acquisition times. This study introduces PI-MoCoNet, a physics-informed motion correction network that integrates spatial and k-space information to remove motion artifacts without explicit motion parameter estimation, enhancing image fidelity and diagnostic reliability. Materials and Methods: PI-MoCoNet consists of a motion detection network (U-net with spatial averaging) to identify corrupted k-space lines and a motion correction network (U-net with Swin Transformer blocks) to reconstruct motion-free images. The correction is guided by three loss functions: reconstruction (L1), perceptual (LPIPS), and data consistency (Ldc). Motion artifacts were simulated via rigid phase encoding perturbations and evaluated on IXI and MR-ART datasets against Pix2Pix, CycleGAN, and U-net using PSNR, SSIM, and NMSE. Results: PI-MoCoNet significantly improved image quality. On IXI, for minor artifacts, PSNR increased from 34.15 dB to 45.95 dB, SSIM from 0.87 to 1.00, and NMSE reduced from 0.55% to 0.04%. For moderate artifacts, PSNR improved from 30.23 dB to 42.16 dB, SSIM from 0.80 to 0.99, and NMSE from 1.32% to 0.09%. For heavy artifacts, PSNR rose from 27.99 dB to 36.01 dB, SSIM from 0.75 to 0.97, and NMSE decreased from 2.21% to 0.36%. On MR-ART, PI-MoCoNet achieved PSNR gains of ~10 dB and SSIM improvements of up to 0.20, with NMSE reductions of ~6%. Ablation studies confirmed the importance of data consistency and perceptual losses, yielding a 1 dB PSNR gain and 0.17% NMSE reduction. Conclusions: PI-MoCoNet effectively mitigates motion artifacts in brain MRI, outperforming existing methods. Its ability to integrate spatial and k-space information makes it a promising tool for clinical use in motion-prone settings. Code: https://github.com/mosaf/PI-MoCoNet.git.
The properties of black holes and accretion flows can be inferred by fitting Event Horizon Telescope (EHT) data to simulated images generated through general relativistic ray tracing (GRRT). However, due to the computationally intensive nature of GRRT, the efficiency of generating specific radiation flux images needs to be improved. This paper introduces the Branch Correction Denoising Diffusion Model (BCDDM), a deep learning framework that synthesizes black hole images directly from physical parameters. The model incorporates a branch correction mechanism and a weighted mixed loss function to enhance accuracy and stability. We have constructed a dataset of 2,157 GRRT-simulated images for training the BCDDM, which spans seven key physical parameters of the radiatively inefficient accretion flow (RIAF) model. Our experiments show a strong correlation between the generated images and their physical parameters. By enhancing the GRRT dataset with BCDDM-generated images and using ResNet50 for parameter regression, we achieve significant improvements in parameter prediction performance. BCDDM offers a novel approach to reducing the computational costs of black hole image generation, providing a faster and more efficient pathway for dataset augmentation, parameter estimation, and model fitting.
The development and validation of automated driving vehicles requires an intensive testing process that can only be performed efficiently with the support of virtual simulators. In the simulation process, the vehicle dynamics behaviour influences the testing of controllers and decision algorithms. Although the use of vehicle dynamics models is well described in the literature, parameter estimation is usually a costly process. In order to achieve a straightforward estimation of the parameters, this work proposes a combination of static and dynamic measurements and the implementation of optimization methods to estimate the parameters. Initially, the vehicle mass and position are calculated from the vehicle weight and the tyre parameters are measured on a dedicated test bench. The transmission, torque converter and other tyre parameters are calculated from data collected during dynamic maneuvers. Finally, a differential evolution method is applied to minimise the errors between the model and the data collected from the real vehicle when subjected to the same control inputs. This work makes an important contribution to accelerate the virtualisation of vehicle dynamics, providing the community with a comprehensive open-source vehicle dynamics model. Furthermore, the proposed methodology has proven to be an efficient way to estimate vehicle dynamics parameters and can be used to test specific maneuvers in the simulated environment.
Given the ubiquity of streaming data, online algorithms have been widely used for parameter estimation, with second-order methods particularly standing out for their efficiency and robustness. In this paper, we study an online sketched Newton method that leverages a randomized sketching technique to perform an approximate Newton step in each iteration, thereby eliminating the computational bottleneck of second-order methods. While existing studies have established the asymptotic normality of sketched Newton methods, a consistent estimator of the limiting covariance matrix remains an open problem. We propose a fully online covariance matrix estimator that is constructed entirely from the Newton iterates and requires no matrix factorization. Compared to covariance estimators for first-order online methods, our estimator for second-order methods is batch-free. We establish the consistency and convergence rate of our estimator, and coupled with asymptotic normality results, we can then perform online statistical inference for the model parameters based on sketched Newton methods. We also discuss the extension of our estimator to constrained problems, and demonstrate its superior performance on regression problems as well as benchmark problems in the CUTEst set.