Understanding the Failure Modes of Out-of-Distribution Generalization
Vaishnavh Nagarajan, Anders Andreassen, Behnam Neyshabur
TL;DR
The paper analyzes why ERM-based training fails to generalize to out-of-distribution domains even on easy tasks where invariant features fully determine the label. It identifies two fundamental failure modes—geometric skew and statistical skew— arising from spurious correlations and develops a gradient-descent-based linear-model framework to isolate and bound these effects. The authors derive theoretical insights and demonstrate them with MNIST/CIFAR-based experiments across simple models and modern architectures, showing that removing skews eliminates OoD failures in these settings. The work argues for a multifaceted approach to OoD generalization, as no single remedy addresses all identified failure modes, and lays a foundation for future algorithmic development in this area.
Abstract
Empirical studies suggest that machine learning models often rely on features, such as the background, that may be spuriously correlated with the label only during training time, resulting in poor accuracy during test-time. In this work, we identify the fundamental factors that give rise to this behavior, by explaining why models fail this way {\em even} in easy-to-learn tasks where one would expect these models to succeed. In particular, through a theoretical study of gradient-descent-trained linear classifiers on some easy-to-learn tasks, we uncover two complementary failure modes. These modes arise from how spurious correlations induce two kinds of skews in the data: one geometric in nature, and another, statistical in nature. Finally, we construct natural modifications of image classification datasets to understand when these failure modes can arise in practice. We also design experiments to isolate the two failure modes when training modern neural networks on these datasets.