Non-normal Limits and Joint Calibration of Cross-fitted Heterogeneity Tests

원문 초록

Cross-fitting is used to test whether a ranking learned in a randomized experiment separates units that benefit from treatment from those that do not: each fold is ranked by a model fitted to the other folds, and the standardized difference in estimated effects between units ranked high and low is compared with a normal distribution. Under the null hypothesis of no heterogeneity the ranking is determined by noise, and because each fold helps to rank the others the fold estimates are dependent. We show that this dependence changes the shape of the null distribution, not just its variance. For least squares on one covariate the limit is bimodal for two folds and skewed for many, no variance correction restores the level, and the model is a counterexample to a published central limit theorem. With many covariates the limit approaches normality, but its variance stays inflated. Removing the non-normal component of the statistic removes its power, so we keep the statistic and change its reference distribution: joint calibration simulates together the moments that choose the rankings and those that evaluate them and applies the same selection to the draws; it has asymptotically exact level and requires no refitting. Applications to clinical and marketing trials, together with random treatment reassignments on their covariates, demonstrate that the standard test substantially inflates Type I error and can produce spurious rejections that joint calibration avoids.