2D Stability Selection: Design Jittering to Exclude Correlated Irrelevant Variables

원문 초록

Stability Selection finds the variables that are chosen consistently across random subsamples of the data, but it can consistently choose the wrong ones. An irrelevant variable that is strongly correlated with relevant variables can be selected in many subsamples, because resampling the observations does not change the variables themselves. We call such a variable a confounder; the term refers only to this correlation and not to causal confounding. We propose 2D Stability Selection, which also adds random noise to the design matrix, at increasing noise levels, and keeps the variables that are selected throughout. On average, the added noise acts as a ridge penalty, so each fit is a noisy elastic net. This changes the conditions under which the Lasso selects the correct variables: it can restore them when a confounder would otherwise be chosen, and it can also make selection worse. For uncorrelated relevant variables we give an exact criterion for the noise levels at which the population elastic-net irrepresentable condition holds, recovery guarantees for a single fit and for the aggregated procedure, and, for thresholds above $1/2$, a bound on the expected number of false positives (under i.i.d.\ observations; its closed form also needs exchangeable irrelevant variables). In simulations, the method excludes the confounders that Stability Selection includes. In the simulated design, an elastic net used without any noise, with a fixed ridge weight or along a deterministic path, excludes the confounders as well, which suggests that for the Lasso the ridge effect, not the randomness of the noise, is responsible.