PyTorchKR Papers

최신 논문

총 1,907편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2504.14969

Evaluating LLMs on Chinese Topic Constructions: A Research Proposal Inspired by Tian et al. (2024)

This paper proposes a framework for evaluating large language models (LLMs) on Chinese topic constructions, focusing on their sensitivity to island constraints. Drawing inspiration from Tian et al. (2024), we outline an experimental design for testing LLMs' grammatical knowledge of Mandarin syntax. While no experiments have been conducted yet, this proposal aims to provide a foundation for future studies and invites feedback on the methodology.

arXiv:2504.13728

Sensitivity analysis enlightens effects of connectivity in a Neural Mass Model under Control-Target mode

Biophysical models of human brain represent the latter as a graph of inter-connected neural regions. Building from the model by Naskar et al. (Network Neuroscience 2021), our motivation was to understand how these brain regions can be connected at neural level to implement some inhibitory control, which calls for inhibitory connectivity rarely considered in such models. In this model, regions are made of inter-connected excitatory and inhibitory pools of neurons, but are long-range connected only via excitatory pools (mutual excitation). We thus extend this model by generalizing connectivity, and we analyse how connectivity affects the behaviour of this model. Focusing on the simplest paradigm made of a Control area and a Target area, we explore four typical kinds of connectivity: mutual excitation, Target inhibition by Control, Control inhibition by Target, and mutual inhibition. For this, we build an analytical sensitivity framework, nesting up sensitivities of isolated pools, of isolated regions, and of the full system. We show that inhibitory control can emerge only in Target inhibition by Control and mutual inhibition connectivities. We next offer an analysis of how the model sensitivities depends on connectivity structure, depending on a parameter controling the strength of the self-inhibition within Target region. Finally, we illustrate the effect of connectivity structure upon control effectivity in response to an external forcing in the Control area. Beyond the case explored here, our methodology to build analytical sensitivities by nesting up levels (pool, region, system) lays the groundwork for expressing nested sensitivities for more complex network configurations, either for this model or any other one.

arXiv:2504.13475

LLM Sensitivity Evaluation Framework for Clinical Diagnosis

Large language models (LLMs) have demonstrated impressive performance across various domains. However, for clinical diagnosis, higher expectations are required for LLM's reliability and sensitivity: thinking like physicians and remaining sensitive to key medical information that affects diagnostic reasoning, as subtle variations can lead to different diagnosis results. Yet, existing works focus mainly on investigating the sensitivity of LLMs to irrelevant context and overlook the importance of key information. In this paper, we investigate the sensitivity of LLMs, i.e. GPT-3.5, GPT-4, Gemini, Claude3 and LLaMA2-7b, to key medical information by introducing different perturbation strategies. The evaluation results highlight the limitations of current LLMs in remaining sensitive to key medical information for diagnostic decision-making. The evolution of LLMs must focus on improving their reliability, enhancing their ability to be sensitive to key information, and effectively utilizing this information. These improvements will enhance human trust in LLMs and facilitate their practical application in real-world scenarios. Our code and dataset are available at https://github.com/chenwei23333/DiagnosisQA.

arXiv:2504.12082

Selective Demonstration Retrieval for Improved Implicit Hate Speech Detection

Hate speech detection is a crucial area of research in natural language processing, essential for ensuring online community safety. However, detecting implicit hate speech, where harmful intent is conveyed in subtle or indirect ways, remains a major challenge. Unlike explicit hate speech, implicit expressions often depend on context, cultural subtleties, and hidden biases, making them more challenging to identify consistently. Additionally, the interpretation of such speech is influenced by external knowledge and demographic biases, resulting in varied detection results across different language models. Furthermore, Large Language Models often show heightened sensitivity to toxic language and references to vulnerable groups, which can lead to misclassifications. This over-sensitivity results in false positives (incorrectly identifying harmless statements as hateful) and false negatives (failing to detect genuinely harmful content). Addressing these issues requires methods that not only improve detection precision but also reduce model biases and enhance robustness. To address these challenges, we propose a novel method, which utilizes in-context learning without requiring model fine-tuning. By adaptively retrieving demonstrations that focus on similar groups or those with the highest similarity scores, our approach enhances contextual comprehension. Experimental results show that our method outperforms current state-of-the-art techniques. Implementation details and code are available at TBD.

arXiv:2504.11576

Effective dimensionality reduction for Greeks computation using Randomized QMC

Global sensitivity analysis is employed to evaluate the effective dimension reduction achieved through Chebyshev interpolation and the conditional pathwise method for Greek estimation of discretely monitored barrier options and arithmetic average Asian options. We compare results from finite difference and Monte Carlo methods with those obtained by using randomized Quasi Monte Carlo combined with Brownian bridge discretization. Additionally, we investigate the benefits of incorporating importance sampling with either the finite difference or Chebyshev interpolation methods. Our findings demonstrate that the reduced effective dimensionality identified through global sensitivity analysis explains the performance advantages of one approach over another. Specifically, the increased smoothness provided by Chebyshev or conditional pathwise methods enhances the convergence rate of randomized Quasi Monte Carlo integration, leading to the significant increase of accuracy and reduced computational costs.

arXiv:2504.12408공식 코드 · Papers with Code

A Human-AI Comparative Analysis of Prompt Sensitivity in LLM-Based Relevance Judgment

Large Language Models (LLMs) are increasingly used to automate relevance judgments for information retrieval (IR) tasks, often demonstrating agreement with human labels that approaches inter-human agreement. To assess the robustness and reliability of LLM-based relevance judgments, we systematically investigate impact of prompt sensitivity on the task. We collected prompts for relevance assessment from 15 human experts and 15 LLMs across three tasks~ -- ~binary, graded, and pairwise~ -- ~yielding 90 prompts in total. After filtering out unusable prompts from three humans and three LLMs, we employed the remaining 72 prompts with three different LLMs as judges to label document/query pairs from two TREC Deep Learning Datasets (2020 and 2021). We compare LLM-generated labels with TREC official human labels using Cohen's $\kappa$ and pairwise agreement measures. In addition to investigating the impact of prompt variations on agreement with human labels, we compare human- and LLM-generated prompts and analyze differences among different LLMs as judges. We also compare human- and LLM-generated prompts with the standard UMBRELA prompt used for relevance assessment by Bing and TREC 2024 Retrieval Augmented Generation (RAG) Track. To support future research in LLM-based evaluation, we release all data and prompts at https://github.com/Narabzad/prompt-sensitivity-relevance-judgements/.

arXiv:2504.13116

Predicting BVD Re-emergence in Irish Cattle From Highly Imbalanced Herd-Level Data Using Machine Learning Algorithms

Bovine Viral Diarrhoea (BVD) has been the focus of a successful eradication programme in Ireland, with the herd-level prevalence declining from 11.3% in 2013 to just 0.2% in 2023. As the country moves toward BVD freedom, the development of predictive models for targeted surveillance becomes increasingly important to mitigate the risk of disease re-emergence. In this study, we evaluate the performance of a range of machine learning algorithms, including binary classification and anomaly detection techniques, for predicting BVD-positive herds using highly imbalanced herd-level data. We conduct an extensive simulation study to assess model performance across varying sample sizes and class imbalance ratios, incorporating resampling, class weighting, and appropriate evaluation metrics (sensitivity, positive predictive value, F1-score and AUC values). Random forests and XGBoost models consistently outperformed other methods, with the random forest model achieving the highest sensitivity and AUC across scenarios, including real-world prediction of 2023 herd status, correctly identifying 219 of 250 positive herds while halving the number of herds that require compared to a blanket-testing strategy.

arXiv:2504.12851

Optimal Capital Structure for Life Insurance Companies Offering Surplus Participation

We adapt Leland's dynamic capital structure model to the context of an insurance company selling participating life insurance contracts explaining the existence of life insurance contracts which provide both a guaranteed payment and surplus participation to the policyholders. Our derivation of the optimal participation rate reveals its pronounced sensitivity to the contract duration and the associated tax rate. Moreover, the asset substitution effect, which describes the tendency of equity holders to increase the riskiness of a company's investment decisions, decreases when adding surplus participation.

arXiv:2504.11720공식 코드 · Papers with Code

Polarisation-Inclusive Spiking Neural Networks for Real-Time RFI Detection in Modern Radio Telescopes

Radio Frequency Interference (RFI) is a known growing challenge for radio astronomy, intensified by increasing observatory sensitivity and prevalence of orbital RFI sources. Spiking Neural Networks (SNNs) offer a promising solution for real-time RFI detection by exploiting the time-varying nature of radio observation and neuron dynamics together. This work explores the inclusion of polarisation information in SNN-based RFI detection, using simulated data from the Hydrogen Epoch of Reionisation Array (HERA) instrument and provides power usage estimates for deploying SNN-based RFI detection on existing neuromorphic hardware. Preliminary results demonstrate state-of-the-art detection accuracy and highlight possible extensive energy-efficiency gains.

arXiv:2504.10350

Benchmarking 3D Human Pose Estimation Models Under Occlusions

This paper addresses critical challenges in 3D Human Pose Estimation (HPE) by analyzing the robustness and sensitivity of existing models to occlusions, camera position, and action variability. Using a novel synthetic dataset, BlendMimic3D, which includes diverse scenarios with multi-camera setups and several occlusion types, we conduct specific tests on several state-of-the-art models. Our study focuses on the discrepancy in keypoint formats between common datasets such as Human3.6M, and 2D datasets such as COCO, commonly used for 2D detection models and frequently input of 3D HPE models. Our work explores the impact of occlusions on model performance and the generality of models trained exclusively under standard conditions. The findings suggest significant sensitivity to occlusions and camera settings, revealing a need for models that better adapt to real-world variability and occlusion scenarios. This research contributed to ongoing efforts to improve the fidelity and applicability of 3D HPE systems in complex environments.

arXiv:2504.11402공식 코드 · Papers with Code

Complex multiannual cycles of Mycoplasma pneumoniae: persistence and the role of stochasticity

The epidemiological dynamics of Mycoplasma pneumoniae are characterized by complex and poorly understood multiannual cycles, posing challenges for forecasting. Using Bayesian methods to fit a seasonally forced transmission model to long-term surveillance data from Denmark (1958-1995, 2010-2025), we investigate the mechanisms driving recurrent outbreaks of M. pneumoniae. The period of the multiannual cycles (predominantly approx. 5 years in Denmark) are explained as a consequence of the interaction of two time-scales in the system, one intrinsic and one extrinsic (seasonal). While it provides an excellent fit to shorter time series (a few decades), we find that the deterministic model eventually settles into an annual cycle, failing to reproduce the observed 4-5-year periodicity long-term. Upon further analysis, the system is found to exhibit transient chaos and thus high sensitivity to stochasticity. We show that environmental (but not purely demographic) stochasticity can sustain the multi-year cycles via stochastic resonance. The disruptive effects of COVID-19 non-pharmaceutical interventions (NPIs) on M. pneumoniae circulation constitute a natural experiment on the effects of large perturbations. Consequently, the effects of NPIs are included in the model and medium-term predictions are explored. Our findings highlight the intrinsic sensitivity of M. pneumoniae dynamics to perturbations and interventions, underscoring the limitations of deterministic epidemic models for long-term prediction. More generally, our results emphasize the potential role of stochasticity as a driver of complex cycles across endemic and recurring pathogens.

arXiv:2504.11319

Sensitivity Analysis of State Space Models for Scrap Composition Estimation in EAF and BOF

This study develops and analyzes linear and nonlinear state space models for estimating the elemental composition of scrap steel used in steelmaking, with applications to Electric Arc Furnace (EAF) and Basic Oxygen Furnace (BOF) processes. The models incorporate mass balance equations and are fitted using a modified Kalman filter for linear cases and the Unscented Kalman Filter (UKF) for nonlinear cases. Using Cu and Cr as representative elements, we assess the sensitivity of model predictions to measurement noise in key process variables, including steel mass, steel composition, scrap input mass, slag mass, and iron oxide fraction in slag. Results show that the models are robust to moderate noise levels in most variables, particularly when errors are below $10\%$. However, accuracy significantly deteriorates with noise in slag mass estimation. These findings highlight the practical feasibility and limitations of applying state space models for real-time scrap composition estimation in industrial settings.

arXiv:2504.08469

Artifact detection and localization in single-channel mobile EEG for sleep research using deep learning and attention mechanisms

Artifacts in the electroencephalogram (EEG) degrade signal quality and impact the analysis of brain activity. Current methods for detecting artifacts in sleep EEG rely on simple threshold-based algorithms that require manual intervention, which is time-consuming and impractical due to the vast volume of data that novel mobile recording systems generate. We propose a convolutional neural network (CNN) model incorporating a convolutional block attention module (CNN-CBAM) to detect and identify the location of artifacts in the sleep EEG with attention maps. We benchmarked this model against six other machine learning and signal processing approaches. We trained/tuned all models on 72 manually annotated EEG recordings obtained during home-based monitoring from 18 healthy participants with a mean (SD) age of 68.05 y ($\pm$5.02). We tested them on 26 separate recordings from 6 healthy participants with a mean (SD) age of 68.33 y ($\pm$4.08), with contained artifacts in 4\% of epochs. CNN-CBAM achieved the highest area under the receiver operating characteristic curve (0.88), sensitivity (0.81), and specificity (0.86) when compared to the other approaches. The attention maps from CNN-CBAM localized artifacts within the epoch with a sensitivity of 0.71 and specificity of 0.67. This work demonstrates the feasibility of automating the detection and localization of artifacts in wearable sleep EEG.

arXiv:2504.08086공식 코드 · Papers with Code

Differentially Private Selection using Smooth Sensitivity

Differentially private selection mechanisms offer strong privacy guarantees for queries aiming to identify the top-scoring element r from a finite set R, based on a dataset-dependent utility function. While selection queries are fundamental in data science, few mechanisms effectively ensure their privacy. Furthermore, most approaches rely on global sensitivity to achieve differential privacy (DP), which can introduce excessive noise and impair downstream inferences. To address this limitation, we propose the Smooth Noisy Max (SNM) mechanism, which leverages smooth sensitivity to yield provably tighter (upper bounds on) expected errors compared to global sensitivity-based methods. Empirical results demonstrate that SNM is more accurate than state-of-the-art differentially private selection methods in three applications: percentile selection, greedy decision trees, and random forests.

arXiv:2504.07278

A Multi-Phase Analysis of Blood Culture Stewardship: Machine Learning Prediction, Expert Recommendation Assessment, and LLM Automation

Blood cultures are often over ordered without clear justification, straining healthcare resources and contributing to inappropriate antibiotic use pressures worsened by the global shortage. In study of 135483 emergency department (ED) blood culture orders, we developed machine learning (ML) models to predict the risk of bacteremia using structured electronic health record (EHR) data and provider notes via a large language model (LLM). The structured models AUC improved from 0.76 to 0.79 with note embeddings and reached 0.81 with added diagnosis codes. Compared to an expert recommendation framework applied by human reviewers and an LLM-based pipeline, our ML approach offered higher specificity without compromising sensitivity. The recommendation framework achieved sensitivity 86%, specificity 57%, while the LLM maintained high sensitivity (96%) but over classified negatives, reducing specificity (16%). These findings demonstrate that ML models integrating structured and unstructured data can outperform consensus recommendations, enhancing diagnostic stewardship beyond existing standards of care.

arXiv:2504.05743

Causal Portfolio Optimization: Principles and Sensitivity-Based Solutions

Fundamental and necessary principles for achieving efficient portfolio optimization based on asset and diversification dynamics are presented. The Commonality Principle is a necessary and sufficient condition for identifying optimal drivers of a portfolio in terms of its diversification dynamics. The proof relies on the Reichenbach Common Cause Principle, along with the fact that the sensitivities of portfolio constituents with respect to the common causal drivers are themselves causal. A conformal map preserves idiosyncratic diversification from the unconditional setting while optimizing systematic diversification on an embedded space of these sensitivities. Causal methodologies for combinatorial driver selection are presented, such as the use of Bayesian networks and correlation-based algorithms from Reichenbach's principle. Limitations of linear models in capturing causality are discussed, and included for completeness alongside more advanced models such as neural networks. Portfolio optimization methods are presented that map risk from the sensitivity space to other risk measures of interest. Finally, the work introduces a novel risk management framework based on Common Causal Manifolds, including both theoretical development and experimental validation. The sensitivity space is predicted along the common causal manifold, which is modeled as a causal time system. Sensitivities are forecasted using SDEs calibrated to data previously extracted from neural networks to move along the manifold via its tangent bundles. An optimization method is then proposed that accumulates information across future predicted tangent bundles on the common causal time system manifold. It aggregates sensitivity-based distance metrics along the trajectory to build a comprehensive sensitivity distance matrix. This matrix enables trajectory-wide optimal diversification, taking into account future dynamics.

arXiv:2504.05196

Universal Lymph Node Detection in Multiparametric MRI with Selective Augmentation

Robust localization of lymph nodes (LNs) in multiparametric MRI (mpMRI) is critical for the assessment of lymphadenopathy. Radiologists routinely measure the size of LN to distinguish benign from malignant nodes, which would require subsequent cancer staging. Sizing is a cumbersome task compounded by the diverse appearances of LNs in mpMRI, which renders their measurement difficult. Furthermore, smaller and potentially metastatic LNs could be missed during a busy clinical day. To alleviate these imaging and workflow problems, we propose a pipeline to universally detect both benign and metastatic nodes in the body for their ensuing measurement. The recently proposed VFNet neural network was employed to identify LN in T2 fat suppressed and diffusion weighted imaging (DWI) sequences acquired by various scanners with a variety of exam protocols. We also use a selective augmentation technique known as Intra-Label LISA (ILL) to diversify the input data samples the model sees during training, such that it improves its robustness during the evaluation phase. We achieved a sensitivity of $\sim$83\% with ILL vs. $\sim$80\% without ILL at 4 FP/vol. Compared with current LN detection approaches evaluated on mpMRI, we show a sensitivity improvement of $\sim$9\% at 4 FP/vol.

arXiv:2504.03943

Multi-Variable Batch Bayesian Optimization in Materials Research: Synthetic Data Analysis of Noise Sensitivity and Problem Landscape Effects

Bayesian Optimization (BO) machine learning method is increasingly used to guide experimental optimization tasks in materials science. To emulate the large number of input variables and noise-containing results in experimental materials research, we perform batch BO simulation of six design variables with a range of noise levels. Two test cases relevant for materials science problems are examined: a needle-in-a-haystack case (Ackley function) that may be encountered in, e.g., molecule optimizations, and a smooth landscape with a local optimum in addition to the global optimum (Hartmann function) that may be encountered in, e.g., material composition optimization. We show learning curves, performance metrics, and visualization to effectively track the optimization progression and evaluate how the optimization outcomes are affected by noise, batch-picking method, choice of acquisition function, and exploration hyperparameter values. We find that the effects of noise depend on the problem landscape: noise degrades the optimization results of a needle-in-a-haystack search (Ackley) dramatically more. However, with increasing noise, we observe an increasing probability of landing on the local optimum in Hartmann. Therefore, prior knowledge of the problem domain structure and noise level is essential when designing BO for materials research experiments. Synthetic data studies -- with known ground truth and controlled noise levels -- enable us to isolate and evaluate the impact of different batch BO components, {\it e.g.}, acquisition policy, objective metrics, and hyperparameter values, before transitioning to the inherent uncertainties of real experimental systems. The results and methodology of this study will facilitate a greater utilization of BO in guiding experimental materials research, specifically in settings with a large number of design variables to optimize.

arXiv:2504.07788

Generalized Passivity Sensitivity Methodology for Small-Signal Stability Analysis

This paper proposes a generalized passivity sensitivity analysis for power system stability studies. The method uncovers the most effective instability mitigation actions for both device-level and system-level investigations. The particular structure of the admittance and nodal models is exploited in the detailed derivation of the passivity sensitivity expressions. These proposed sensitivities are validated for different parameters at device-level and at system-level. Compared to previous stability and sensitivity methods, it does not require detailed system information, such as exact system eigenvalues, while it provides valuable information for a less conservative stable system design. In addition, we demonstrate how to utilize the proposed method through case studies with different converter controls and system-wide insights showing its general applicability.

arXiv:2504.05924공식 코드 · Papers with Code

A Control-Oriented Simplified Single Particle Model with Grouped Parameter and Sensitivity Analysis for Lithium-Ion Batteries

Lithium-ion batteries are widely used in transportation, energy storage, and consumer electronics, driving the need for reliable battery management systems (BMS) for state estimation and control. The Single Particle Model (SPM) balances computational efficiency and accuracy but faces challenges in parameter estimation due to numerous parameters. Current SPM models using parabolic approximation introduce intermediate variables and hard to do parameter grouping. This study presents a control-oriented SPM reformulation that employs parameter grouping and parabolic approximation to simplify model parameters while using average and surface lithium-ion concentrations as model output. By parameter grouping, the original 17 parameters were reduced to 9 grouped parameters. The reformulated model achieves a reduced-order ordinary differential equation form while maintaining mathematical accuracy equivalent to the pre-grouped discretized SPM. Through Sobol sensitivity analysis under various current profiles, the grouped parameters were reduced from 9 to 6 highly sensitive parameters. Results demonstrate that estimating these 6 parameters achieves comparable practical accuracy to estimating all 9 parameters, with faster convergence. This control-oriented SPM enhances BMS applications by facilitating state estimation and control while reducing parameter estimation requirements.