We address the problem of uncertainty quantification (UQ) in the localization of a sound source within adverse acoustic environments. Estimating the position of the source is influenced by various factors, such as noise and reverberation, leading to significant uncertainty. Quantifying this uncertainty is essential, particularly when localization outcomes impact critical decision-making processes, such as in robot audition, where the accuracy of location estimates directly influences subsequent actions. Despite this, common localization methods offer point estimates without quantifying the estimation uncertainty. To address this, we employ conformal prediction (CP)-a framework that delivers statistically valid prediction intervals (PIs) with finite-sample guarantees, independent of the data distribution. However, commonly used Inductive CP (ICP) methods require a large amount of labeled data, which can be difficult to obtain in the localization setting. To mitigate this limitation, we incorporate a semi-supervised manifold-based localization method using Gaussian process regression (GPR), with an efficient Transductive CP (TCP) technique, specifically designed for GPR. We demonstrate that our method generates statistically valid PIs across different acoustic conditions, while producing smaller intervals compared to baselines.
We report initial work towards constructing ecologically valid benchmarks to assess the capabilities of large multimodal models for engaging in situated collaboration. In contrast to existing benchmarks, in which question-answer pairs are generated post hoc over preexisting or synthetic datasets via templates, human annotators, or large language models (LLMs), we propose and investigate an interactive system-driven approach, where the questions are generated by users in context, during their interactions with an end-to-end situated AI system. We illustrate how the questions that arise are different in form and content from questions typically found in existing embodied question answering (EQA) benchmarks and discuss new real-world challenge problems brought to the fore.
In our previous research, we provided a reasoning system (called LeSAC) based on argumentation theory to provide legal support to designers during the design process. Building on this, this paper explores how to provide designers with effective explanations for their legally relevant design decisions. We extend the previous system for providing explanations by specifying norms and the key legal or ethical principles for justifying actions in normative contexts. Considering that first-order logic has strong expressive power, in the current paper we adopt a first-order deontic logic system with deontic operators and preferences. We illustrate the advantages and necessity of introducing deontic logic and designing explanations under LeSAC by modelling two cases in the context of autonomous driving. In particular, this paper also discusses the requirements of the updated LeSAC to guarantee rationality, and proves that a well-defined LeSAC can satisfy the rationality postulate for rule-based argumentation frameworks. This ensures the system's ability to provide coherent, legally valid explanations for complex design decisions.
A pandemic is the spread of a disease across large regions, and can have devastating costs to the society in terms of health, economic and social. As such, the study of effective pandemic mitigation strategies can yield significant positive impact on the society. A pandemic can be mathematically described using a compartmental model, such as the Susceptible Infected Removed (SIR) model. In this paper, we extend the solution equations of the SIR model to a state transition model with lockdowns. We formalize a metric hybrid planning problem based on this state transition model, and solve it using a metric hybrid planner. We improve the runtime effectiveness of the metric hybrid planner with the addition of valid inequalities, and demonstrate the success of our approach both theoretically and experimentally under various challenging settings.
Multi-agent strategies have emerged as a promising approach to enhance the reasoning abilities of Large Language Models (LLMs) by assigning specialized roles in the problem-solving process. Concurrently, Tree of Thoughts (ToT) methods have shown potential in improving reasoning for complex question-answering tasks by exploring diverse reasoning paths. A critical limitation in multi-agent reasoning is the 'Reasoner' agent's shallow exploration of reasoning paths. While ToT strategies could help mitigate this problem, they may generate flawed reasoning branches, which could harm the trustworthiness of the final answer. To leverage the strengths of both multi-agent reasoning and ToT strategies, we introduce a novel approach combining ToT-based Reasoner agents with a Thought Validator agent. Multiple Reasoner agents operate in parallel, employing ToT to explore diverse reasoning paths. The Thought Validator then scrutinizes these paths, considering a Reasoner's conclusion only if its reasoning is valid. This method enables a more robust voting strategy by discarding faulty reasoning paths, enhancing the system's ability to tackle tasks requiring systematic and trustworthy reasoning. Our method demonstrates superior performance compared to existing techniques when evaluated on the GSM8K dataset, outperforming the standard ToT strategy by an average 5.6% across four LLMs. The code and related content can be found in: https://github.com/SecureAIAutonomyLab/MA-ToT
Inequalities may appear in many models. They can be as simple as assuming a parameter is nonnegative, possibly a regression coefficient or a treatment effect. This paper focuses on the case that there is only one inequality and proposes a confidence interval that is particularly attractive, called the inequality-imposed confidence interval (IICI). The IICI is simple. It does not require simulations or tuning parameters. The IICI is adaptive. It reduces to the usual confidence interval (calculated by adding and subtracting the standard error times the $1 - \alpha/2$ standard normal quantile) when the inequality is sufficiently slack. When the inequality is sufficiently violated, the IICI reduces to an equality-imposed confidence interval (the usual confidence interval for the submodel where the inequality holds with equality). Also, the IICI is uniformly valid and has (weakly) shorter length than the usual confidence interval; it is never longer. The first empirical application considers a linear regression when a coefficient is known to be nonpositive. A second empirical application considers an instrumental variables regression when the endogeneity of a regressor is known to be nonnegative.
When conducting causal inference or designing policy, researchers are often concerned with the existence and extent of interference between units, which may be influenced by factors such as distance, proximity, and connection strength. However, complex correlations across units pose significant challenges for inference. This paper introduces partial null randomization tests (PNRTs), a novel framework for testing interference in experimental settings. PNRTs adopt a design-based approach, combining unconditional randomization testing with pairwise comparisons to enable straightforward implementation and ensure finite-sample validity under minimal assumptions about network structure. To illustrate the method's broad applicability, this paper applies it to a large-scale experiment by Blattman et al. (2021) in Bogota, Colombia, which evaluates the impact of hotspot policing on crime using street segments as units of analysis. The findings indicate that increasing police patrolling time in hotspots has a significant displacement effect on violent crime but not on property crime. A simulation study calibrated to this dataset further demonstrates the strong power properties of PNRTs and their suitability for general interference scenarios.
Recent advancements in deep learning have led to widespread use of techniques for audio content generation, notably employing Denoising Diffusion Probabilistic Models (DDPM) across various tasks. Among these, Foley Sound Synthesis is of particular interest for its role in applications for the creation of multimedia content. Given the temporal-dependent nature of sound, it is crucial to design generative models that can effectively handle the sequential modeling of audio samples. Selective State Space Models (SSMs) have recently been proposed as a valid alternative to previously proposed techniques, demonstrating competitive performance with lower computational complexity. In this paper, we introduce MambaFoley, a diffusion-based model that, to the best of our knowledge, is the first to leverage the recently proposed SSM known as Mamba for the Foley sound generation task. To evaluate the effectiveness of the proposed method, we compare it with a state-of-the-art Foley sound generative model using both objective and subjective analyses.
Even if the depth maps captured by RGB-D sensors deployed in real environments are often characterized by large areas missing valid depth measurements, the vast majority of depth completion methods still assumes depth values covering all areas of the scene. To address this limitation, we introduce SteeredMarigold, a training-free, zero-shot depth completion method capable of producing metric dense depth, even for largely incomplete depth maps. SteeredMarigold achieves this by using the available sparse depth points as conditions to steer a denoising diffusion probabilistic model. Our method outperforms relevant top-performing methods on the NYUv2 dataset, in tests where no depth was provided for a large area, achieving state-of-art performance and exhibiting remarkable robustness against depth map incompleteness. Our source code is publicly available at https://steeredmarigold.github.io.
In time-series classification, understanding model decisions is crucial for their application in high-stakes domains such as healthcare and finance. Counterfactual explanations, which provide insights by presenting alternative inputs that change model predictions, offer a promising solution. However, existing methods for generating counterfactual explanations for time-series data often struggle with balancing key objectives like proximity, sparsity, and validity. In this paper, we introduce TX-Gen, a novel algorithm for generating counterfactual explanations based on the Non-dominated Sorting Genetic Algorithm II (NSGA-II). TX-Gen leverages evolutionary multi-objective optimization to find a diverse set of counterfactuals that are both sparse and valid, while maintaining minimal dissimilarity to the original time series. By incorporating a flexible reference-guided mechanism, our method improves the plausibility and interpretability of the counterfactuals without relying on predefined assumptions. Extensive experiments on benchmark datasets demonstrate that TX-Gen outperforms existing methods in generating high-quality counterfactuals, making time-series models more transparent and interpretable.
We analyze the methodological approach and validity of interpretation of using national-level time-series regression analyses relating epidemic outcomes to policies that estimate many models involving permutations of analytic choices (i.e., a "multiverse" approach). Specifically, we evaluate the possible biases and pitfalls of interpretation of a multiverse approach to the context of government responses to epidemics using the example of COVID-19 and a recently published peer-reviewed paper by Bendavid and Patel (2024) along with the subsequent commentary that the two authors published discussing and interpreting the implications of their work. While we identify multiple potential errors and sources of biases in how the specific analyses were undertaken that are also relevant for other studies employing similar approaches, our most important finding involves constructing a counterexample showing that causal model specification-agnostic multiverse analyses can be incorrectly used to suggest that no consistent effect can be discovered in data especially in cases where most specifications estimated with the data are far from causally valid. Finally, we suggest an alternative approach involving hypothesis-drive specifications that explicitly account for infectiousness across jurisdictions in the analysis as well as the interconnected ways that policies and behavioral responses may evolve within and across these jurisdictions.
Partial least squares (PLS) is a simple factorisation method that works well with high dimensional problems in which the number of observations is limited given the number of independent variables. In this article, we show that PLS can perform better than ordinary least squares (OLS), least absolute shrinkage and selection operator (LASSO) and ridge regression in forecasting quarterly gross domestic product (GDP) growth, covering the period from 2000 to 2023. In fact, through dimension reduction, PLS proved to be effective in lowering the out-of-sample forecasting error, specially since 2020. For the period 2000-2019, the four methods produce similar results, suggesting that PLS is a valid regularisation technique like LASSO or ridge.
Dynamic Stochastic General Equilibrium (DSGE) models are nowadays a crucial quantitative tool for policy-makers. However, they did not emerge spontaneously. They are built upon previously established ideas in Economics and relatively recent advancements in Mathematics. This essay provides a comprehensive coverage of their history, starting from the pioneering Neoclassical general equilibrium theories and eventually reaching the New Neoclassical Synthesis (NNS). In addition, the mathematical tools involved in formulating a DSGE model are thoroughly presented. I argue that this history has a mixed nature rather than an absolutist or relativist one, that the NNS may have emerged due to the complementary nature of New Classical and New Keynesian theories, and that the recent adoption and development of DSGE models by central banks from different countries has entailed a departure from the goal of building a universally valid theory that Economics has always had. The latter means that DSGE modeling has landed not without loss of generality.
When it comes to structural estimation of risk preferences from data on choices, random utility models have long been one of the standard research tools in economics. A recent literature has challenged these models, pointing out some concerning monotonicity and, thus, identification problems. In this paper, we take a second look and point out that some of the criticism - while extremely valid - may have gone too far, demanding monotonicity of choice probabilities in decisions where it is not so clear whether it should be imposed. We introduce a new class of random utility models based on carefully constructed generalized risk premia which always satisfy our relaxed monotonicity criteria. Moreover, we show that some of the models used in applied research like the certainty-equivalent-based random utility model for CARA utility actually lie in this class of monotonic stochastic choice models. We conclude that not all random utility models are bad.
The information bottleneck (IB) problem is a widely studied framework in machine learning for extracting compressed features that are informative for downstream tasks. However, current approaches to solving the IB problem rely on a heuristic tuning of hyperparameters, offering no guarantees that the learned features satisfy information-theoretic constraints. In this work, we introduce a statistically valid solution to this problem, referred to as IB via multiple hypothesis testing (IB-MHT), which ensures that the learned features meet the IB constraints with high probability, regardless of the size of the available dataset. The proposed methodology builds on Pareto testing and learn-then-test (LTT), and it wraps around existing IB solvers to provide statistical guarantees on the IB constraints. We demonstrate the performance of IB-MHT on classical and deterministic IB formulations, including experiments on distillation of language models. The results validate the effectiveness of IB-MHT in outperforming conventional methods in terms of statistical robustness and reliability.
Path planners that can interpret free-form natural language instructions hold promise to automate a wide range of robotics applications. These planners simplify user interactions and enable intuitive control over complex semi-autonomous systems. While existing symbolic approaches offer guarantees on the correctness and efficiency, they struggle to parse free-form natural language inputs. Conversely, neural approaches based on pre-trained Large Language Models (LLMs) can manage natural language inputs but lack performance guarantees. In this paper, we propose a neuro-symbolic framework for path planning from natural language inputs called NSP. The framework leverages the neural reasoning abilities of LLMs to i) craft symbolic representations of the environment and ii) a symbolic path planning algorithm. Next, a solution to the path planning problem is obtained by executing the algorithm on the environment representation. The framework uses a feedback loop from the symbolic execution environment to the neural generation process to self-correct syntax errors and satisfy execution time constraints. We evaluate our neuro-symbolic approach using a benchmark suite with 1500 path-planning problems. The experimental evaluation shows that our neuro-symbolic approach produces 90.1% valid paths that are on average 19-77% shorter than state-of-the-art neural approaches.
In this paper, our goal is to investigate to what degree multilingual pretrained language models capture cross-linguistically valid abstract linguistic representations. We take the approach of developing curated synthetic data on a large scale, with specific properties, and using them to study sentence representations built using pretrained language models. We use a new multiple-choice task and datasets, Blackbird Language Matrices (BLMs), to focus on a specific grammatical structural phenomenon -- subject-verb agreement across a variety of sentence structures -- in several languages. Finding a solution to this task requires a system detecting complex linguistic patterns and paradigms in text representations. Using a two-level architecture that solves the problem in two steps -- detect syntactic objects and their properties in individual sentences, and find patterns across an input sequence of sentences -- we show that despite having been trained on multilingual texts in a consistent manner, multilingual pretrained language models have language-specific differences, and syntactic structure is not shared, even across closely related languages.
Generative models for level generation have shown great potential in game production. However, they often provide limited control over the generation, and the validity of the generated levels is unreliable. Despite this fact, only a few approaches that learn from existing data provide the users with ways of controlling the generation, simultaneously addressing the generation of unsolvable levels. %One of the main challenges it faces is that levels generated through automation may not be solvable thus requiring validation. are not always engaging, challenging, or even solvable. This paper proposes Avalon, a novel method to improve models that learn from existing level designs using difficulty statistics extracted from gameplay. In particular, we use a conditional variational autoencoder to generate layouts for match-3 levels, conditioning the model on pre-collected statistics such as game mechanics like difficulty and relevant visual features like size and symmetry. Our method is general enough that multiple approaches could potentially be used to generate these statistics. We quantitatively evaluate our approach by comparing it to an ablated model without difficulty conditioning. Additionally, we analyze both quantitatively and qualitatively whether the style of the dataset is preserved in the generated levels. Our approach generates more valid levels than the same method without difficulty conditioning.
Object pose distribution estimation is crucial in robotics for better path planning and handling of symmetric objects. Recent distribution estimation approaches employ contrastive learning-based approaches by maximizing the likelihood of a single pose estimate in the absence of a CAD model. We propose a pose distribution estimation method leveraging symmetry respecting correspondence distributions and shape information obtained using a CAD model. Contrastive learning-based approaches require an exhaustive amount of training images from different viewpoints to learn the distribution properly, which is not possible in realistic scenarios. Instead, we propose a pipeline that can leverage correspondence distributions and shape information from the CAD model, which are later used to learn pose distributions. Besides, having access to pose distribution based on correspondences before learning pose distributions conditioned on images, can help formulate the loss between distributions. The prior knowledge of distribution also helps the network to focus on getting sharper modes instead. With the CAD prior, our approach converges much faster and learns distribution better by focusing on learning sharper distribution near all the valid modes, unlike contrastive approaches, which focus on a single mode at a time. We achieve benchmark results on SYMSOL-I and T-Less datasets.
The exponential growth in data sizes and storage costs has brought considerable challenges to the data science community, requiring solutions to run learning methods on such data. While machine learning has scaled to achieve predictive accuracy in big data settings, statistical inference and uncertainty quantification tools are still lagging. Priority scientific fields collect vast data to understand phenomena typically studied with statistical methods like regression. In this setting, regression parameter estimation can benefit from efficient computational procedures, but the main challenge lies in computing error process parameters with complex covariance structures. Identifying and estimating these structures is essential for inference and often used for uncertainty quantification in machine learning with Gaussian Processes. However, estimating these structures becomes burdensome as data scales, requiring approximations that compromise the reliability of outputs. These approximations are even more unreliable when complexities like long-range dependencies or missing data are present. This work defines and proves the statistical properties of the Generalized Method of Wavelet Moments with Exogenous variables (GMWMX), a highly scalable, stable, and statistically valid method for estimating and delivering inference for linear models using stochastic processes in the presence of data complexities like latent dependence structures and missing data. Applied examples from Earth Sciences and extensive simulations highlight the advantages of the GMWMX.