A Law of Data Reconstruction for Random Features (and Beyond)

TL;DR

This paper investigates data memorization from a reconstruction perspective and establishes a law of reconstructibility: in the random features (RF) regression setting, the entire training dataset can be recovered from model parameters once the parameter count satisfies $p \gg d n$. The authors prove two identifiability results (Theorem reconveronelargen and Theorem reconveralln2) and derive a practical reconstruction objective based on the orthogonal projector to the feature-span, which they optimize to recover the inputs. They validate the theory numerically across RF models, two-layer networks, and deep ResNets, demonstrating the threshold at $p \approx d n$ and successful data recovery at larger $p$. The work also discusses sign ambiguities under certain activations and highlights potential privacy implications of over-parameterization, suggesting directions for future analysis in the intermediate regime $n \ll p \ll d n$.

Abstract

Large-scale deep learning models are known to memorize parts of the training set. In machine learning theory, memorization is often framed as interpolation or label fitting, and classical results show that this can be achieved when the number of parameters $p$ in the model is larger than the number of training samples $n$. In this work, we consider memorization from the perspective of data reconstruction, demonstrating that this can be achieved when $p$ is larger than $dn$, where $d$ is the dimensionality of the data. More specifically, we show that, in the random features model, when $p \gg dn$, the subspace spanned by the training samples in feature space gives sufficient information to identify the individual samples in input space. Our analysis suggests an optimization method to reconstruct the dataset from the model parameters, and we demonstrate that this method performs well on various architectures (random features, two-layer fully-connected and deep residual networks). Our results reveal a law of data reconstruction, according to which the entire training dataset can be recovered as $p$ exceeds the threshold $dn$.

Figures (17)

• Figure 1: Thresholds for label fitting and data reconstruction in the random features model. (Left) We consider RF regression with ReLU activation on binary labels (frogs vs. trucks) with $n=100$ images (50 examples per class) from CIFAR-10 ($d=3072$). We report the mean for both the reconstruction error (in red, defined in (\ref{['eq:recerror']})) and the training error (in black, mean squared error), as the number of parameters $p$ increases. Statistics are computed across 4 distinct random seeds, and the standard deviation across seeds is very small (the confidence interval at one standard deviation is reported in the plot as shaded area, but it is imperceptible). For $p \geq n$, training labels are memorized, while reconstruction is feasible when $p$ becomes larger than $dn$. (Right) Results of the reconstruction when $p=10dn$. Odd rows report the ground truth images, while even rows the reconstructed ones which are all visually very similar.
• Figure 2: Features of the training dataset $\Phi$ are spanned by the features of the reconstructed dataset $\hat{\Phi}$. We consider the same setup as in Figure \ref{['fig:teaser']}. For different values of $p$, we optimize until $\mathcal{L}(\hat{X}) = 0$, and report reconstruction error (in red, defined in Eq. (\ref{['eq:recerror']})) and normalized residual $\|P^\perp_{\hat{\Phi}} \varphi(x_i)\|_2$ averaged over $i \in [n]$ (in blue), with their confidence interval at one standard deviation (shaded area). Further details and evidence are in Appendix \ref{['sec:add_exp_span']}, see Figure \ref{['fig:residuals_rfsynthd=100n=20_2layermulti_resnet_cifar10_n=10']}.
• Figure 3: Thresholds for label fitting and data reconstruction when training on i.i.d. data unformly drawn from the $d$-dimensional sphere. We consider RF regression with ReLU activation, fitting a noisy linear model. We report mean (solid line) and standard deviation (shaded area) for both the reconstruction error (in red) and training loss (in black) as the number of parameters $p$ increases, at different choices of input dimensions $d$ and number of dataset examples $n$. Statistics are computed across 10 distinct random seeds. Two distinct thresholds clearly emerge: $p\gg n$ for label fitting, and $p\gg dn$ for data reconstruction.
• Figure 4: Images reconstructed from an RF model with ReLU activation may have the wrong sign. We repeat the experiment of Figure \ref{['fig:teaser']} using a different random seed and observe that reconstructions from ReLU models can appear as sign-flipped versions of original training data. This is due to the fact that ReLU has odd Hermite coefficients of order $\ge 3$ equal to zero, as discussed in Remark \ref{['rmk:sign']}.
• Figure 5: Reconstruction of higher-dimensional images. We repeat the same experiment of Figure \ref{['fig:teaser']} (right) using $n = 20$ samples from Tiny-ImageNet and a random features model with $p=10dn$.

Theorems & Definitions (26)

• Theorem 1
• Remark 1: Sign ambiguity
• Theorem 2
• Remark 2: Technical challenge for $n \geq 3$
• Lemma B.1
• proof
• Lemma B.2
• proof
• Lemma B.3
• proof