Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization

원문 초록

Deep neural networks can learn salient features from an in-distribution (ID) training window without identifying the data-generating structure needed for out-of-distribution (OOD) generalization. Understanding this limitation requires examining the identifiability of the data-generating process (DGP). From a single training window, the DGP that yields the correct OOD extrapolation is not identifiable: infinitely many DGPs are observationally equivalent within a fixed tolerance on that window yet diverge arbitrarily beyond it, and no ID criterion alone can reliably distinguish them. A structural commitment---the feature map, label map, and model class $(\varphi, \psi, \mathcal{M})$---specifies the assumed DGP and governs OOD generalization while leaving ID performance essentially unchanged. When architecture, pretraining, data augmentation, input formats, or domain knowledge implicitly supply the correct commitment, the model generalizes beyond the training window. When it cannot infer OOD-relevant structure from ID evidence, it fails to generalize. When the commitment is correct {\em and} identifiable, OOD error vanishes. We demonstrate this mechanism in models trained from scratch and in foundation models across four natural-science settings (chemistry, physics, biology, and histopathology), as well as in a positional-encoding study spanning Transformer, Mamba, and S4D architectures. Finally, a controlled study shows that correct features are necessary but not sufficient for OOD generalization.