Generative Modeling of Weights: Generalization or Memorization?

TL;DR

This paper interrogates whether four prominent generative models of neural network weights (Hyper-Representations, G.pt, HyperDiffusion, and P-diff) can produce novel weights or merely memorize training checkpoints. Through analyses of weight-space proximity, model behavior, and novelty metrics such as IoU-based misprediction similarity, the authors show that generated weights largely reproduce or interpolate existing checkpoints, offering little advantage over simple baselines like noise addition or averaging. They show memorization persists across data regimes and architectures, with limited evidence of genuine generalization; data scaling helps in some cases (notably G.pt) but not others, and symmetry-based priors via augmentation are insufficient without architectural integration. The study highlights the importance of evaluating memorization in weight-domain models and suggests future work should focus on data modality-specific priors and symmetry-aware designs to achieve true novelty in weight generation. Overall, the findings urge caution in applying weight-generating models beyond replication and emphasize the need for principled evaluation and modality-aware modeling.

Abstract

Generative models have recently been explored for synthesizing neural network weights. These approaches take neural network checkpoints as training data and aim to generate high-performing weights during inference. In this work, we examine four representative, well-known methods on their ability to generate novel model weights, i.e., weights that are different from the checkpoints seen during training. Contrary to claims in prior work, we find that these methods synthesize weights largely by memorization: they produce either replicas, or, at best, simple interpolations of the training checkpoints. Moreover, they fail to outperform simple baselines, such as adding noise to the weights or taking a simple weight ensemble, in obtaining different and simultaneously high-performing models. Our further analysis suggests that this memorization might result from limited data, overparameterized models, and the underuse of structural priors specific to weight data. These findings highlight the need for more careful design and rigorous evaluation of generative models when applied to new domains. Our code is available at https://github.com/boyazeng/weight_memorization.

Figures (24)

• Figure 1: Building on their success in image generation, generative models have recently been applied to synthesize weights for neural networks. While they can produce effective neural network checkpoints (e.g., classification models with high accuracy), it is unclear whether they can generalize beyond the training set to generate novel weights.
• Figure 2: Generated weights highly resemble training weights. For each method, we display three heatmaps, showing weight values for 64 randomly selected parameter indices. In each heatmap, the top row (outlined in red) shows the values of a random generated checkpoint, and the three rows below (separated by white lines) show its three nearest training checkpoints. We observe that for every generated checkpoint, at least one training checkpoint is nearly identical to it.
• Figure 3: Reconstructing SVHN netzer2011reading classification model weights with Hyper-Representations' autoencoder degrades model performance.
• Figure 4: Generated checkpoints are closer to training checkpoints than training checkpoints are to one another, except for p-diff. This indicates that generated weights have lower novelty than a new model trained from scratch. The red and blue histograms represent the distributions of the $L_2$ distances to the nearest training checkpoints (excluding self-comparisons).
• Figure 5: Generated models produce highly similar outputs to their nearest training models. Each row shows the decision boundaries or reconstructed 3D shapes of a randomly selected generated checkpoint (“generated”) and its nearest training checkpoint (“training”). For p-diff models trained on CIFAR-100, decision boundaries are shown for ten randomly selected classes.