Active vision enables dynamic visual perception, offering an alternative to static feedforward architectures in computer vision, which rely on large datasets and high computational resources. Biological selective attention mechanisms allow agents to focus on salient Regions of Interest (ROIs), reducing computational demand while maintaining real-time responsiveness. Event-based cameras, inspired by the mammalian retina, enhance this capability by capturing asynchronous scene changes enabling efficient low-latency processing. To distinguish moving objects while the event-based camera is in motion the agent requires an object motion segmentation mechanism to accurately detect targets and center them in the visual field (fovea). Integrating event-based sensors with neuromorphic algorithms represents a paradigm shift, using Spiking Neural Networks to parallelize computation and adapt to dynamic environments. This work presents a Spiking Convolutional Neural Network bioinspired attention system for selective attention through object motion sensitivity. The system generates events via fixational eye movements using a Dynamic Vision Sensor integrated into the Speck neuromorphic hardware, mounted on a Pan-Tilt unit, to identify the ROI and saccade toward it. The system, characterized using ideal gratings and benchmarked against the Event Camera Motion Segmentation Dataset, reaches a mean IoU of 82.2% and a mean SSIM of 96% in multi-object motion segmentation. The detection of salient objects reaches 88.8% accuracy in office scenarios and 89.8% in low-light conditions on the Event-Assisted Low-Light Video Object Segmentation Dataset. A real-time demonstrator shows the system's 0.12 s response to dynamic scenes. Its learning-free design ensures robustness across perceptual scenes, making it a reliable foundation for real-time robotic applications serving as a basis for more complex architectures.
Transformers, with their attention mechanisms, have emerged as the state-of-the-art architectures of sequential modeling and empirically outperform feed-forward neural networks (FFNNs) across many fields, such as natural language processing and computer vision. However, their generalization ability, particularly for low-sensitivity functions, remains less studied. We bridge this gap by analyzing transformers on the $k$-parity problem. Daniely and Malach (NeurIPS 2020) show that FFNNs with one hidden layer and $O(nk^7 \log k)$ parameters can learn $k$-parity, where the input length $n$ is typically much larger than $k$. In this paper, we prove that FFNNs require at least $\Omega(n)$ parameters to learn $k$-parity, while transformers require only $O(k)$ parameters, surpassing the theoretical lower bound needed by FFNNs. We further prove that this parameter efficiency cannot be achieved with fixed attention heads. Our work establishes transformers as theoretically superior to FFNNs in learning parity function, showing how their attention mechanisms enable parameter-efficient generalization in functions with low sensitivity.
To understand the complexity of sequence classification tasks, Hahn et al. (2021) proposed sensitivity as the number of disjoint subsets of the input sequence that can each be individually changed to change the output. Though effective, calculating sensitivity at scale using this framework is costly because of exponential time complexity. Therefore, we introduce a Sensitivity-based Multi-Armed Bandit framework (SMAB), which provides a scalable approach for calculating word-level local (sentence-level) and global (aggregated) sensitivities concerning an underlying text classifier for any dataset. We establish the effectiveness of our approach through various applications. We perform a case study on CHECKLIST generated sentiment analysis dataset where we show that our algorithm indeed captures intuitively high and low-sensitive words. Through experiments on multiple tasks and languages, we show that sensitivity can serve as a proxy for accuracy in the absence of gold data. Lastly, we show that guiding perturbation prompts using sensitivity values in adversarial example generation improves attack success rate by 15.58%, whereas using sensitivity as an additional reward in adversarial paraphrase generation gives a 12.00% improvement over SOTA approaches. Warning: Contains potentially offensive content.
Traditional ML models utilize controlled approximations during high loads, employing faster, but less accurate models in a process called accuracy scaling. However, this method is less effective for generative text-to-image models due to their sensitivity to input prompts and performance degradation caused by large model loading overheads. This work introduces a novel text-to-image inference system that optimally matches prompts across multiple instances of the same model operating at various approximation levels to deliver high-quality images under high loads and fixed budgets.
As large language models (LLMs) become integral to diverse applications, ensuring their reliability under varying input conditions is crucial. One key issue affecting this reliability is order sensitivity, wherein slight variations in the input arrangement can lead to inconsistent or biased outputs. Although recent advances have reduced this sensitivity, the problem remains unresolved. This paper investigates the extent of order sensitivity in LLMs whose internal components are hidden from users (such as closed-source models or those accessed via API calls). We conduct experiments across multiple tasks, including paraphrasing, relevance judgment, and multiple-choice questions. Our results show that input order significantly affects performance across tasks, with shuffled inputs leading to measurable declines in output accuracy. Few-shot prompting demonstrates mixed effectiveness and offers partial mitigation; however, fails to fully resolve the problem. These findings highlight persistent risks, particularly in high-stakes applications, and point to the need for more robust LLMs or improved input-handling techniques in future development.
Atherosclerosis is a chronic inflammatory disease of the artery wall. The early stages of atherosclerosis are driven by interactions between lipids and monocyte-derived-macrophages (MDMs). The mechanisms that govern the spatial distribution of lipids and MDMs in the lesion remain poorly understood. In this paper, we develop a spatially-resolved and lipid-structured model for early atherosclerosis. The model development and analysis are guided by images of human coronary lesions by Nakashima et al. 2007. Consistent with their findings, the model predicts that lipid initially accumulates deep in the intima due to a spatially non-uniform LDL retention capacity. The model also qualitatively reproduces the global internal maxima in the Nakashima images only when the MDM mobility is sufficiently sensitive to lipid content, and MDM lifespan sufficiently insensitive. Introducing lipid content-dependence to MDM mobility and mean lifespan produced minimal impact on model behaviour at early times, but strongly impacted lesion composition at steady state. Increases to the sensitivity of MDM lifespan to lipid content yield lesions with fewer MDMs, less total lesion lipid content and reduced mean MDM infiltration depth. Increases to the sensitivity of MDM mobility to lipid content also reduces the MDM infiltration depth, but increases the proportion of lipid-laden MDMs. We find that MDM lipid content increases with spatial depth, regardless of blood LDL and HDL content. These results shed light on the mechanisms that drive spatial variation in the composition of early atherosclerotic lesions, and the role of macrophage lipid content in disease progression.
KV cache quantization can improve Large Language Models (LLMs) inference throughput and latency in long contexts and large batch-size scenarios while preserving LLMs effectiveness. However, current methods have three unsolved issues: overlooking layer-wise sensitivity to KV cache quantization, high overhead of online fine-grained decision-making, and low flexibility to different LLMs and constraints. Therefore, we thoroughly analyze the inherent correlation of layer-wise transformer attention patterns to KV cache quantization errors and study why key cache is more important than value cache for quantization error reduction. We further propose a simple yet effective framework KVTuner to adaptively search for the optimal hardware-friendly layer-wise KV quantization precision pairs for coarse-grained KV cache with multi-objective optimization and directly utilize the offline searched configurations during online inference. To reduce the computational cost of offline calibration, we utilize the intra-layer KV precision pair pruning and inter-layer clustering to reduce the search space. Experimental results show that we can achieve nearly lossless 3.25-bit mixed precision KV cache quantization for LLMs like Llama-3.1-8B-Instruct and 4.0-bit for sensitive models like Qwen2.5-7B-Instruct on mathematical reasoning tasks. The maximum inference throughput can be improved by 38.3% compared with KV8 quantization over various context lengths.
This article presents a Compact Nested Hexagonal Metamaterial Sensor designed for microwave sensing to characterize material permittivity in S and X-band applications. The proposed sensor attained compact dimensions of merely 30 mm x 30 mm x 0.79 mm. This innovative design technique employs a distinctive and compact architecture with elevated electromagnetic (EM) field strength, enhancing the precision of the sensing mechanism in the microwave frequency spectrum. The design geometry and dimensions attained resonance frequencies of 3.98 GHz and 11.57 GHz, with notch depths of -13.16129 dB and -10.23024 dB, respectively. The design evolution, metamaterial properties, equivalent circuit model, and electric (E) is delineated to elucidate the stopband features at the resonant frequency. The suggested sensor attains a very high sensitivity of 9.55% in transmission mode (S21) for a permittivity range of 1 to 6. The reflection and transmission properties of the proposed CRR-based sensor are validated by simulations using the mathematical equations of the design. Furthermore, the sensor's performance is corroborated by utilizing several dielectric materials (Roger R04350B, Roger RT5880 and FR-4). The computed outcomes demonstrate alignment with the simulated results. The compact, low-profile sensor design and its excellent sensitivity for characterizing material permittivity render the suggested sensor appropriate for permittivity sensing applications.
With the growing penetration of distributed renewable energy, distributed control approaches are widely utilized in voltage control of active distribution networks (ADNs), suffering from limited applicability for control devices or heavy communication burden. This paper develops an innovative distributed voltage control strategy of ADNs with global sensitivities (DVC-GS), which integrates network information from a global optimization view to coordinate energy storages, PV inverters and the OLTC with little communication and computing time. The global sensitivities of voltage violations across all buses with respect to nodal voltage and active/reactive power injection are formulated to quantify the ability of each controllable resource for alleviating voltage violations in the entire ADN. The back-and-forth communication combined with the modified DistFlow model is newly developed for the real-time update of global sensitivities at each bus with high accuracy and no iteration. Then the coordinated control of various devices is implemented on basis of global sensitivities to effectively ensure voltage security of ADNs. Comprehensive simulations based on modified 11-bus and 123-bus systems demonstrate the significant efficiency and superiority of the global sensitivities and DVC-GS.
Recent works explore deep learning's success by examining functions or data with hierarchical structure. To study the learning complexity of functions with hierarchical structure, we study the noise stability of functions with tree hierarchical structure on independent inputs. We show that if each function in the hierarchy is $\varepsilon$-far from linear, the noise stability is exponentially small in the depth of the hierarchy. Our results have immediate applications for learning. In the Boolean setting using the results of Dachman-Soled, Feldman, Tan, Wan and Wimmer (2014) our results provide Statistical Query super-polynomial lower bounds for learning classes that are based on hierarchical functions. Similarly, using the results of Diakonikolas, Kane, Pittas and Zarifis (2021) our results provide super-polynomial lower bounds for SQ learning under the Gaussian measure. Using the results of Abbe, Bengio, Cornacchiam, Kleinberg, Lotfi, Raghu and Zhang (2022) our results imply sample complexity lower bounds for learning hierarchical functions with gradient descent on fully connected neural networks.
The Latent Stochastic Differential Equation (SDE) is a powerful tool for time series and sequence modeling. However, training Latent SDEs typically relies on adjoint sensitivity methods, which depend on simulation and backpropagation through approximate SDE solutions, which limit scalability. In this work, we propose SDE Matching, a new simulation-free method for training Latent SDEs. Inspired by modern Score- and Flow Matching algorithms for learning generative dynamics, we extend these ideas to the domain of stochastic dynamics for time series and sequence modeling, eliminating the need for costly numerical simulations. Our results demonstrate that SDE Matching achieves performance comparable to adjoint sensitivity methods while drastically reducing computational complexity.
This paper investigates an underexplored challenge in large language models (LLMs): the impact of KV cache compression methods on LLMs' fundamental capabilities. Although existing methods achieve impressive compression ratios on long-context benchmarks, their effects on core model capabilities remain understudied. We present a comprehensive benchmark KVFundaBench to systematically evaluate the effects of KV cache compression across diverse fundamental LLM capabilities, spanning world knowledge, commonsense reasoning, arithmetic reasoning, code generation, safety, and long-context understanding and generation.Our analysis reveals serval key findings: (1) \textit{Task-Dependent Degradation}; (2) \textit{Model-Type Robustness} (3) \textit{Prompt Length Vulnerability}; (4) \textit{Chunk-Level Superiority}; (5) \textit{Prompt-Gain Sensitivity}; (6) \textit{Long-Context Generation Sensitivity}. Based on our analysis of attention patterns and cross-task compression performance, we propose ShotKV, a novel compression approach that distinctly handles prefill and decoding phases while maintaining shot-level semantic coherence. Empirical results show that ShotKV achieves $9\%$-$18\%$ performance improvements on long-context generation tasks under aggressive compression ratios.
Adaptive optimization algorithms -- such as Adagrad, Adam, and their variants -- have found widespread use in machine learning, signal processing and many other settings. Several methods in this family are not rotationally equivariant, meaning that simple reparameterizations (i.e. change of basis) can drastically affect their convergence. However, their sensitivity to the choice of parameterization has not been systematically studied; it is not clear how to identify a "favorable" change of basis in which these methods perform best. In this paper we propose a reparameterization method and demonstrate both theoretically and empirically its potential to improve their convergence behavior. Our method is an orthonormal transformation based on the expected gradient outer product (EGOP) matrix, which can be approximated using either full-batch or stochastic gradient oracles. We show that for a broad class of functions, the sensitivity of adaptive algorithms to choice-of-basis is influenced by the decay of the EGOP matrix spectrum. We illustrate the potential impact of EGOP reparameterization by presenting empirical evidence and theoretical arguments that common machine learning tasks with "natural" data exhibit EGOP spectral decay.
Pre-biopsy magnetic resonance imaging (MRI) is increasingly used to target suspicious prostate lesions. This has led to artificial intelligence (AI) applications improving MRI-based detection of clinically significant prostate cancer (CsPCa). However, MRI-detected lesions must still be mapped to transrectal ultrasound (TRUS) images during biopsy, which results in missing CsPCa. This study systematically evaluates a multimodal AI framework integrating MRI and TRUS image sequences to enhance CsPCa identification. The study included 3110 patients from three cohorts across two institutions who underwent prostate biopsy. The proposed framework, based on the 3D UNet architecture, was evaluated on 1700 test cases, comparing performance to unimodal AI models that use either MRI or TRUS alone. Additionally, the proposed model was compared to radiologists in a cohort of 110 patients. The multimodal AI approach achieved superior sensitivity (80%) and Lesion Dice (42%) compared to unimodal MRI (73%, 30%) and TRUS models (49%, 27%). Compared to radiologists, the multimodal model showed higher specificity (88% vs. 78%) and Lesion Dice (38% vs. 33%), with equivalent sensitivity (79%). Our findings demonstrate the potential of multimodal AI to improve CsPCa lesion targeting during biopsy and treatment planning, surpassing current unimodal models and radiologists; ultimately improving outcomes for prostate cancer patients.
Timely and accurate assessment of cognitive impairment is a major unmet need in populations at risk. Alterations in speech and language can be early predictors of Alzheimer's disease and related dementias (ADRD) before clinical signs of neurodegeneration. Voice biomarkers offer a scalable and non-invasive solution for automated screening. However, the clinical applicability of machine learning (ML) remains limited by challenges in generalisability, interpretability, and access to patient data to train clinically applicable predictive models. Using DementiaBank recordings (N=291, 64% female), we evaluated ML techniques for ADRD screening and severity prediction from spoken language. We validated model generalisability with pilot data collected in-residence from older adults (N=22, 59% female). Risk stratification and linguistic feature importance analysis enhanced the interpretability and clinical utility of predictions. For ADRD classification, a Random Forest applied to lexical features achieved a mean sensitivity of 69.4% (95% confidence interval (CI) = 66.4-72.5) and specificity of 83.3% (78.0-88.7). On real-world pilot data, this model achieved a mean sensitivity of 70.0% (58.0-82.0) and specificity of 52.5% (39.3-65.7). For severity prediction using Mini-Mental State Examination (MMSE) scores, a Random Forest Regressor achieved a mean absolute MMSE error of 3.7 (3.7-3.8), with comparable performance of 3.3 (3.1-3.5) on pilot data. Linguistic features associated with higher ADRD risk included increased use of pronouns and adverbs, greater disfluency, reduced analytical thinking, lower lexical diversity and fewer words reflecting a psychological state of completion. Our interpretable predictive modelling offers a novel approach for in-home integration with conversational AI to monitor cognitive health and triage higher-risk individuals, enabling earlier detection and intervention.
This paper presents a comprehensive study on a novel multilayer surface plasmon resonance (SPR) biosensor designed for detecting trace-level toxins in liquid samples with exceptional precision and efficiency. Leveraging the Kretschmann configuration, the proposed design integrates advanced two-dimensional materials, including black phosphorus (BP) and transition metal dichalcogenides (TMDs), to significantly enhance the performance metrics of the sensor. Key innovations include the optimization of sensitivity through precise material layering, minimization of full-width at half-maximum (FWHM) to improve signal resolution, and maximization of the figure of merit (FoM) for superior detection accuracy. Numerical simulations are employed to validate the structural and functional enhancements of the biosensor. The results demonstrate improved interaction between the evanescent field and the analyte, enabling detection at trace concentrations with higher specificity. This biosensor is poised to contribute to advancements in biochemical sensing, environmental monitoring, and other critical applications requiring high-sensitivity toxin detection.
Early diagnosis and noninvasive monitoring of neurological disorders require sensitivity to elusive cellular-level alterations that occur much earlier than volumetric changes observable with the millimeter-resolution of medical imaging modalities. Morphological changes in axons, such as axonal varicosities or beadings, are observed in neurological disorders, as well as in development and aging. Here, we reveal the sensitivity of time-dependent diffusion MRI (dMRI) to axonal morphology at the micrometer scale. Scattering theory uncovers the two parameters that determine the diffusive dynamics of water in axons: the average reciprocal cross-section and the variance of long-range cross-sectional fluctuations. This theoretical development allowed us to predict dMRI metrics sensitive to axonal alterations across tens of thousands of axons in seconds rather than months of simulations in a rat model of traumatic brain injury. Our approach bridges the gap between micrometers and millimeters in resolution, offering quantitative, objective biomarkers applicable to a broad spectrum of neurological disorders.
Most empirical works that study the financing decisions of family businesses use financial ratios. These data present asymmetry, non-normality, non-linearity and even dependence on the results of the choice of which accounting figure goes to the numerator and denominator of the ratio. This article uses compositional data analysis (CoDa) as well as classical analysis strategies to compare the structure of balance sheet liabilities between family and non-family businesses, showing the sensitivity of the results to the methodology used. The results prove the need to use appropriate methodologies to advance the academic discipline.
Dengue is a vector borne infectious disease. The disease is transmitted byAedes mosquitoes. In the present work, SEIR - SEI compartmental epidemiological model is used to describe dengue disease transmission dynamics. The human population is classified into four epidemiological states: Susceptible, Exposed, Infectious and Recovered humans. The mosquito population is divided into three epidemiological states: Susceptible, Exposed and Infectious mosquitoes. Associated basic reproduction number which determines whether the disease will persist or die out, is calculated by Next Generation method. Stability of the equilibrium points of the model is analyzed in terms of basic reproduction number. Sensitivity analysis is carried out to study importance of modelparameters.
Background: Quantitative susceptibility mapping (QSM) of the brain is an advanced MRI technique for assessing tissue characteristics based on magnetic susceptibility, which varies with the composition of the tissue, such as iron, calcium, and myelin levels. QSM consists of multiple processing steps, with various choices for each step. Despite its increasing application in detecting and monitoring neurodegenerative diseases, the impact of algorithmic choices in QSM's workflow on clinical outcomes has not been thoroughly quantified. Objective: This study aimed to evaluate how choices in background field removal (BFR), dipole inversion algorithms, and anatomical referencing impact the sensitivity and reproducibility error of QSM in detecting group-level and longitudinal changes in deep gray matter susceptibility in a clinical setting. Methods: We compared 378 different QSM pipelines using a 10-year follow-up dataset of healthy adults. We analyzed the sensitivity of pipelines to detect known aging-related susceptibility changes in the DGM over time. Results: We found high variability in the sensitivity of QSM pipelines to detect susceptibility changes. The study highlighted that while most pipelines could detect changes reliably, the choice of BFR algorithm and the referencing strategy substantially influenced the outcome reproducibility error and sensitivity. Notably, pipelines using RESHARP with AMP-PE, HEIDI or LSQR inversion showed the highest overall sensitivity. Conclusions: The findings underscore the critical influence of algorithmic choices in QSM processing on the accuracy and reliability of detecting physiological changes in the brain. This has profound implications for clinical research and trials where QSM is used as a biomarker for disease progression, highlighting that careful consideration should be given to pipeline configuration to optimize clinical outcomes.