Logistic Regression for Massive Data with Rare Events
원문 초록
This paper studies binary logistic regression for rare events data, or imbalanced data, where the number of events (observations in one class, often called cases) is significantly smaller than the number of nonevents (observations in the other class, often called controls). We first derive the asymptotic distribution of the maximum likelihood estimator (MLE) of the unknown parameter, which shows that the asymptotic variance convergences to zero in a rate of the inverse of the number of the events instead of the inverse of the full data sample size. This indicates that the available information in rare events data is at the scale of the number of events instead of the full data sample size. Furthermore, we prove that under-sampling a small proportion of the nonevents, the resulting under-sampled estimator may have identical asymptotic distribution to the full data MLE. This demonstrates the advantage of under-sampling nonevents for rare events data, because this procedure may significantly reduce the computation and/or data collection costs. Another common practice in analyzing rare events data is to over-sample (replicate) the events, which has a higher computational cost. We show that this procedure may even result in efficiency loss in terms of parameter estimation.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Analytic estimation of parameters of stochastic volatility diffusion models with exponential-affine characteristic function for currency option pricing
A Simple Approximate Bayesian Inference Neural Surrogate for Stochastic Petri Net Models
Robust Spatiotemporal Epidemic Modeling with Integrated Adaptive Outlier Detection
Active Learning for Manifold Gaussian Process Regression
IMC-PINN-FE: A Physics-Informed Neural Network for Patient-Specific Left Ventricular Finite Element Modeling with Image Motion Consistency and Biomechanical Parameter Estimation
Vector Contrastive Learning For Pixel-Wise Pretraining In Medical Vision
토론
이 논문과 연결된 포럼 글이 아직 없습니다.