Trade-offs in Data Memorization via Strong Data Processing Inequalities

TL;DR

This work develops a general, information-theoretic framework that ties strong data processing inequalities (SDPIs) to memorization in learning systems. By introducing dominating variables and approximate SDPIs, the authors derive lower bounds on excess memorization for simple binary classification problems, with sharp trade-offs in Gaussian, Boolean, and sparse Boolean hypercube settings, and extend these results to mixtures of clusters. The approach yields mem_n lower bounds on the order of ${d}/{n}$ (or ${d}/{2^{2n}}$ in certain regimes) that are tight up to logarithmic factors, and provides matching upper bounds via straightforward learning algorithms. The framework also connects to practical concerns in large language models by interpreting natural data as mixtures of clusters in embedding spaces, and it generalizes to multi-cluster mixtures, offering a principled view of how data tail distributions influence memorization costs. Overall, the results advance theoretical understanding of memorization, generalization, and privacy in high-dimensional, long-tailed data regimes, and suggest directions for analyzing broader data-generating processes.

Abstract

Recent research demonstrated that training large language models involves memorization of a significant fraction of training data. Such memorization can lead to privacy violations when training on sensitive user data and thus motivates the study of data memorization's role in learning. In this work, we develop a general approach for proving lower bounds on excess data memorization, that relies on a new connection between strong data processing inequalities and data memorization. We then demonstrate that several simple and natural binary classification problems exhibit a trade-off between the number of samples available to a learning algorithm, and the amount of information about the training data that a learning algorithm needs to memorize to be accurate. In particular, $Ω(d)$ bits of information about the training data need to be memorized when $O(1)$ $d$-dimensional examples are available, which then decays as the number of examples grows at a problem-specific rate. Further, our lower bounds are generally matched (up to logarithmic factors) by simple learning algorithms. We also extend our lower bounds to more general mixture-of-clusters models. Our definitions and results build on the work of Brown et al. (2021) and address several limitations of the lower bounds in their work.

Figures (2)

• Figure 1: Illustration for Theorem \ref{['thm: excess mem']}. Given $\theta\sim\Psi,\,X_{1:n}\sim\mathcal{P}_\theta^n,\,X \sim\mathcal{P}_\theta$, the orange arrows represent two SDPIs, which together necessitate excess memorization on the order of $(1-\tau_n)/\rho_n$.
• Figure 2: Illustration of Theorem \ref{['thm: Z SDPI']}. The blue variables dominate the learning problem, and each blue SDPI implies an (approximate) SDPI in orange, resulting in excess memorization.

Theorems & Definitions (30)

• Definition 3.1
• Definition 3.4
• Proposition 3.5
• Remark 3.6
• Theorem 3.7
• proof : Proof of Theorem \ref{['thm: excess mem']}
• Theorem 3.8
• proof : Proof of Theorem \ref{['thm: Z SDPI']}
• Remark 3.9
• Theorem 4.1