Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyond

Qiongxiu Li, Xiaoyu Luo, Yiyi Chen, Johannes Bjerva

TL;DR

The paper examines memorization as a dual driving force in trustworthy ML within long-tail data, arguing that existing theories conflate noise, atypicality, and class imbalance. It proposes a three-level granularity framework to disentangle memorization's beneficial aspects (e.g., rare-but-useful patterns) from its risks to privacy, fairness, and robustness, and it sustains this with theoretical and empirical perspectives. A core contribution is the systematic taxonomy linking memorization to three granularity levels, alongside proxies, and a case study on multilingual LLMs to illustrate practical implications. The work sets a roadmap for future research, urging granular measurements and refined learning objects (e.g., counterfactual memorization scores) to harmonize performance with societal trust across high-stakes domains.

Abstract

The role of memorization in machine learning (ML) has garnered significant attention, particularly as modern models are empirically observed to memorize fragments of training data. Previous theoretical analyses, such as Feldman's seminal work, attribute memorization to the prevalence of long-tail distributions in training data, proving it unavoidable for samples that lie in the tail of the distribution. However, the intersection of memorization and trustworthy ML research reveals critical gaps. While prior research in memorization in trustworthy ML has solely focused on class imbalance, recent work starts to differentiate class-level rarity from atypical samples, which are valid and rare intra-class instances. However, a critical research gap remains: current frameworks conflate atypical samples with noisy and erroneous data, neglecting their divergent impacts on fairness, robustness, and privacy. In this work, we conduct a thorough survey of existing research and their findings on trustworthy ML and the role of memorization. More and beyond, we identify and highlight uncharted gaps and propose new revenues in this research direction. Since existing theoretical and empirical analyses lack the nuances to disentangle memorization's duality as both a necessity and a liability, we formalize three-level long-tail granularity - class imbalance, atypicality, and noise - to reveal how current frameworks misapply these levels, perpetuating flawed solutions. By systematizing this granularity, we draw a roadmap for future research. Trustworthy ML must reconcile the nuanced trade-offs between memorizing atypicality for fairness assurance and suppressing noise for robustness and privacy guarantee. Redefining memorization via this granularity reshapes the theoretical foundation for trustworthy ML, and further affords an empirical prerequisite for models that align performance with societal trust.

Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyond

TL;DR

The paper examines memorization as a dual driving force in trustworthy ML within long-tail data, arguing that existing theories conflate noise, atypicality, and class imbalance. It proposes a three-level granularity framework to disentangle memorization's beneficial aspects (e.g., rare-but-useful patterns) from its risks to privacy, fairness, and robustness, and it sustains this with theoretical and empirical perspectives. A core contribution is the systematic taxonomy linking memorization to three granularity levels, alongside proxies, and a case study on multilingual LLMs to illustrate practical implications. The work sets a roadmap for future research, urging granular measurements and refined learning objects (e.g., counterfactual memorization scores) to harmonize performance with societal trust across high-stakes domains.

Abstract

The role of memorization in machine learning (ML) has garnered significant attention, particularly as modern models are empirically observed to memorize fragments of training data. Previous theoretical analyses, such as Feldman's seminal work, attribute memorization to the prevalence of long-tail distributions in training data, proving it unavoidable for samples that lie in the tail of the distribution. However, the intersection of memorization and trustworthy ML research reveals critical gaps. While prior research in memorization in trustworthy ML has solely focused on class imbalance, recent work starts to differentiate class-level rarity from atypical samples, which are valid and rare intra-class instances. However, a critical research gap remains: current frameworks conflate atypical samples with noisy and erroneous data, neglecting their divergent impacts on fairness, robustness, and privacy. In this work, we conduct a thorough survey of existing research and their findings on trustworthy ML and the role of memorization. More and beyond, we identify and highlight uncharted gaps and propose new revenues in this research direction. Since existing theoretical and empirical analyses lack the nuances to disentangle memorization's duality as both a necessity and a liability, we formalize three-level long-tail granularity - class imbalance, atypicality, and noise - to reveal how current frameworks misapply these levels, perpetuating flawed solutions. By systematizing this granularity, we draw a roadmap for future research. Trustworthy ML must reconcile the nuanced trade-offs between memorizing atypicality for fairness assurance and suppressing noise for robustness and privacy guarantee. Redefining memorization via this granularity reshapes the theoretical foundation for trustworthy ML, and further affords an empirical prerequisite for models that align performance with societal trust.

Paper Structure

This paper contains 54 sections, 1 theorem, 12 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Preliminaries
  3. Machine learning
  4. Trustworthy Machine Learning
  5. Privacy
  6. Robustness
  7. Fairness
  8. Demographic Parity.
  9. Equalized Odds.
  10. Equal Opportunity.
  11. Transparency and Explainability
  12. Theoretical Foundations of Memorization in ML
  13. Memorization
  14. Empirical Evidence of Memorization
  15. Formal Definition of (Label) Memorization
  16. ...and 39 more sections

Key Result

Theorem 1

For certain natural tasks $q$, every dataset $X$ contains a subset $X_S$ (singleton data) such that $X_S$ has an expected size of $\Omega(n)$, where $n$ is the dataset size. The entropy of $X_S$, conditioned on the data distribution $P$, denoted as $H(X_S | P)$, scaled as $\Omega(nd)$, where $d$ is where $M$ represents the trained model, and $I(X_S; M | P)$ is the mutual information between $X_S$

Figures (2)

  • Figure 1: Sample images of different granularity from CIFAR-10 and CIFAR-100 datasets.
  • Figure 2: An overview of trustworthy attribute trade-off, providing an intuitive trade-off guide to the detailed information in Table \ref{['tab:simplified-defences']}.

Theorems & Definitions (1)

  • Theorem 1