A monitoring framework for deployed machine learning models with supply chain examples
원문 초록
Actively monitoring machine learning models during production operations helps ensure prediction quality and detection and remediation of unexpected or undesired conditions. Monitoring models already deployed in big data environments brings the additional challenges of adding monitoring in parallel to the existing modelling workflow and controlling resource requirements. In this paper, we describe (1) a framework for monitoring machine learning models; and, (2) its implementation for a big data supply chain application. We use our implementation to study drift in model features, predictions, and performance on three real data sets. We compare hypothesis test and information theoretic approaches to drift detection in features and predictions using the Kolmogorov-Smirnov distance and Bhattacharyya coefficient. Results showed that model performance was stable over the evaluation period. Features and predictions showed statistically significant drifts; however, these drifts were not linked to changes in model performance during the time of our study.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Lite-RVFL: A Lightweight Random Vector Functional-Link Neural Network for Learning Under Concept Drift
Describing Nonstationary Data Streams in Frequency Domain
Early Concept Drift Detection via Prediction Uncertainty
Structuring the Processing Frameworks for Data Stream Evaluation and Application
SUDS: A Strategy for Unsupervised Drift Sampling
A method to benchmark high-dimensional process drift detection
토론
이 논문과 연결된 포럼 글이 아직 없습니다.