Decoding Generalization from Memorization in Deep Neural Networks

TL;DR

The paper addresses why overparameterized deep networks memorize noisy data yet still generalize. It introduces MASC, a Minimum Angle Subspace Classifier that decodes generalization from layerwise representations by constructing class-conditioned subspaces via PCA and choosing the class whose subspace projection minimizes the angle with the layer output. Across multiple model-family and dataset settings, the results show latent generalization persists in memorized models, and MASC often surpasses the model’s own test performance, even when training labels are heavily corrupted. The findings suggest that generalization can be decoded from internal representations, with implications for boosting robustness to label noise and informing theories of generalization in deep learning.

Abstract

Overparameterized Deep Neural Networks that generalize well have been key to the dramatic success of Deep Learning in recent years. The reasons for their remarkable ability to generalize are not well understood yet. It has also been known that deep networks possess the ability to memorize training data, as evidenced by perfect or high training accuracies on models trained with corrupted data that have class labels shuffled to varying degrees. Concomitantly, such models are known to generalize poorly, i.e. they suffer from poor test accuracies, due to which it is thought that the act of memorizing substantially degrades the ability to generalize. It has, however, been unclear why the poor generalization that accompanies such memorization, comes about. One possibility is that in the process of training with corrupted data, the layers of the network irretrievably reorganize their representations in a manner that makes generalization difficult. The other possibility is that the network retains significant ability to generalize, but the trained network somehow chooses to readout in a manner that is detrimental to generalization. Here, we provide evidence for the latter possibility by demonstrating, empirically, that such models possess information in their representations for substantially improved generalization, even in the face of memorization. Furthermore, such generalization abilities can be easily decoded from the internals of the trained model, and we build a technique to do so from the outputs of specific layers of the network. We demonstrate results on multiple models trained with a number of standard datasets.

Figures (39)

• Figure 1: Minimum Angle Subspace Classifier (MASC) accuracy over the layers of the network when the data is projected onto corrupted training subspaces with the indicated corruption degree, for multiple models/datasets. Rows corresponds to plots with the same corruption degree & the columns correspond to the models, as noted. Training accuracy (dashed line) & testing accuracy (dotted line) of the model is shown. FC corresponds to fully connected layer with $ReLU$ activation whereas Flat corresponds to flatten layer without $ReLU$ activation. The number of class-wise PCA components of these models are shown in Figure \ref{['pca_appendix_angle1']} in section \ref{['number_pca']} of the Appendix.
• Figure 2: Minimum Angle Subspace Classifier (MASC) accuracy over the layers of the network when the data set is projected onto corrupted subspace and subspace corresponding to true training labels. Rows corresponds to plots which have the same corruption degree and the columns correspond to the models as noted. Training and testing accuracy of the model is shown. FC corresponds to fully connected layer with ReLU activation whereas Flat corresponds to flatten layer without ReLU activation. The respective number of class-wise PCA components for true training label subspaces of the models is shown in Figure \ref{['pca_appendix_angle2']} in section \ref{['number_pca']} of the Appendix.
• Figure 3: Minimum Angle Subspace Classifier (MASC) accuracy over the layers of the generalized network when the data set is projected onto corrupted training subspaces with the indicated corruption degree. Rows corresponds to plots which have the same corruption degree & the columns correspond to the generalized models as noted. Training & testing accuracy of the generalized model is shown. FC corresponds to fully connected layer with $ReLU$ activation whereas Flat corresponds to flatten layer without $ReLU$ activation. The respective number of class-wise PCA components of the models is shown in Figure \ref{['pca_appendix_angle3']} in Section \ref{['number_pca']} of the Appendix.
• Figure 4: Class-conditioned training data subspaces on layerwise outputs of MLP using PCA. Top: Schematic of MLP model used in the work. Bottom: Creating the class-conditioned training subspace for ReLU (128) layer where 128 are the number of neurons.
• Figure 5: MASC test accuracy over the layers of the network when the data is projected onto corrupted training subspaces with the indicated corruption degree. Best Model Testing Accuracy corresponds accuracy of the testing data of the model if early stopping was used.