On Local Overfitting and Forgetting in Deep Neural Networks

TL;DR

This work reframes overfitting in deep learning through local forgetting, introducing the forget fraction $F_e$ and forgetting time $\hat{n}$ to quantify how training can erase correct classifications in subregions of the data space even as overall test performance rises. It develops a theory using over-parameterized deep linear networks to characterize forgotten knowledge and links forgetting to spectral data properties, especially projections onto leading principal components. Based on these insights, the paper proposes KnowledgeFusion (KF), a post-training ensemble method that combines mid-training and final models (with a small window around a peak forget fraction) and optionally uses self-distillation to avoid inference costs, achieving consistent improvements across datasets and architectures, including under label noise and transfer learning. Empirical results show KF often matches or surpasses more expensive ensembles with substantially lower cost, supporting the practical value of recovering forgotten knowledge and offering a new tool for improving generalization in modern networks.

Abstract

The infrequent occurrence of overfitting in deep neural networks is perplexing: contrary to theoretical expectations, increasing model size often enhances performance in practice. But what if overfitting does occur, though restricted to specific sub-regions of the data space? In this work, we propose a novel score that captures the forgetting rate of deep models on validation data. We posit that this score quantifies local overfitting: a decline in performance confined to certain regions of the data space. We then show empirically that local overfitting occurs regardless of the presence of traditional overfitting. Using the framework of deep over-parametrized linear models, we offer a certain theoretical characterization of forgotten knowledge, and show that it correlates with knowledge forgotten by real deep models. Finally, we devise a new ensemble method that aims to recover forgotten knowledge, relying solely on the training history of a single network. When combined with self-distillation, this method enhances the performance of any trained model without adding inference costs. Extensive empirical evaluations demonstrate the efficacy of our method across multiple datasets, contemporary neural network architectures, and training protocols.

Figures (11)

• Figure 1: Local overfitting and forgetting in a binary problem, where blue and orange denote the different classes, and circles mark the validation set. The initial (left) and final (right) separators of a hypothetical learning method are shown, where $\otimes$ marks prediction errors. Clearly the final classifier has a smaller generalization error, but now one point at the top is 'forgotten'.
• Figure 2: (a)-(b): Blue denotes test accuracy. Among those correctly recognized in each epoch $e$, orange denotes the fraction that remains correctly recognized at the end. The test accuracy (the blue curve) shows a clear double ascent of accuracy, which is much less pronounced in the orange curve. During the decrease in test accuracy - the range of epochs between the first and second dashed red vertical lines - the large gap between the blue and orange plots indicates the fraction of test data that has been correctly learned in the first ascent and then forgotten, without ever being re-learned in the later recovery period of the second ascent. (c)-(d): The difference between the number of clean and noisy datapoints at each epoch during the second ascent of test accuracy (the epochs after the second dashed red vertical line), counting datapoints with large loss only. Positive (negative) value indicates that clean (noisy) datapoints are more dominant in the corresponding epoch.
• Figure 3: (a) The $F_e$ score (\ref{['eq:forget']}) of ConvNeXt trained on Imagenet, 3 network sizes: small $\rightarrow$ blue, base $\rightarrow$ orange and large $\rightarrow$ green. Accuracy remained consistent across all network sizes, while it is evident that $F_e$ increases with the network size. (b-c) Within the set of wrongly classified test points after training, we show (b) the fraction that was correctly predicted (y-axis) for x epochs (x-axis), and (c) the last epoch in which an example was classified correctly.
• Figure 4: Empirical results, correlating the sets of examples forgotten during the training of a DNN and those forgotten during the training of a deep linear network (see text for details). Note in (d) that early on, roughly $\frac{1}{6}$ of the points to be forgotten by our deep model are also forgotten by the deep linear model.
• Figure 5: The forget fraction $F_e$, as defined in (\ref{['eq:forget']}), of common neural networks trained on various image classification datasets and different architectures.

Theorems & Definitions (4)

• Definition 1: Forget time
• Claim 1
• proof
• Definition 2