An Empirical Study of Uncertainty Estimation Techniques for Detecting Drift in Data Streams
원문 초록
In safety-critical domains such as autonomous driving and medical diagnosis, the reliability of machine learning models is crucial. One significant challenge to reliability is concept drift, which can cause model deterioration over time. Traditionally, drift detectors rely on true labels, which are often scarce and costly. This study conducts a comprehensive empirical evaluation of using uncertainty values as substitutes for error rates in detecting drifts, aiming to alleviate the reliance on labeled post-deployment data. We examine five uncertainty estimation methods in conjunction with the ADWIN detector across seven real-world datasets. Our results reveal that while the SWAG method exhibits superior calibration, the overall accuracy in detecting drifts is not notably impacted by the choice of uncertainty estimation method, with even the most basic method demonstrating competitive performance. These findings offer valuable insights into the practical applicability of uncertainty-based drift detection in real-world, safety-critical applications.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
A Framework for Uncertainty Quantification Based on Nearest Neighbors Across Layers
Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization
LifelongPR: Lifelong knowledge fusion for point cloud place recognition based on replay and prompt learning
$I^{2}$-World: Intra-Inter Tokenization for Efficient Dynamic 4D Scene Forecasting
MCAM: Multimodal Causal Analysis Model for Ego-Vehicle-Level Driving Video Understanding
AI-Driven Cytomorphology Image Synthesis for Medical Diagnostics
토론
이 논문과 연결된 포럼 글이 아직 없습니다.