Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Easy Data Unlearning Bench

Roy Rinberg, Pol Puigdemont, Martin Pawelczyk, Volkan Cevher

TL;DR

This work introduces a unified and extensible benchmarking suite that simplifies the evaluation of unlearning algorithms using the KLoM (KL divergence of Margins) metric, and enables reproducible, scalable, and fair comparison across unlearning methods.

Abstract

Evaluating machine unlearning methods remains technically challenging, with recent benchmarks requiring complex setups and significant engineering overhead. We introduce a unified and extensible benchmarking suite that simplifies the evaluation of unlearning algorithms using the KLoM (KL divergence of Margins) metric. Our framework provides precomputed model ensembles, oracle outputs, and streamlined infrastructure for running evaluations out of the box. By standardizing setup and metrics, it enables reproducible, scalable, and fair comparison across unlearning methods. We aim for this benchmark to serve as a practical foundation for accelerating research and promoting best practices in machine unlearning. Our code and data are publicly available.

Easy Data Unlearning Bench

TL;DR

This work introduces a unified and extensible benchmarking suite that simplifies the evaluation of unlearning algorithms using the KLoM (KL divergence of Margins) metric, and enables reproducible, scalable, and fair comparison across unlearning methods.

Abstract

Evaluating machine unlearning methods remains technically challenging, with recent benchmarks requiring complex setups and significant engineering overhead. We introduce a unified and extensible benchmarking suite that simplifies the evaluation of unlearning algorithms using the KLoM (KL divergence of Margins) metric. Our framework provides precomputed model ensembles, oracle outputs, and streamlined infrastructure for running evaluations out of the box. By standardizing setup and metrics, it enables reproducible, scalable, and fair comparison across unlearning methods. We aim for this benchmark to serve as a practical foundation for accelerating research and promoting best practices in machine unlearning. Our code and data are publicly available.
Paper Structure (17 sections, 2 equations, 2 figures)

This paper contains 17 sections, 2 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Scope.
  3. Acknowledgements.
  4. Metrics
  5. Problems with Existing Metrics: U-LiRA
  6. KLoM
  7. Assumptions and Robustness
  8. Hyperparameters and Practical Estimation
  9. Future Directions: Extending to LLMs
  10. Teacher-forcing KLoM
  11. Benchmark
  12. Included Datasets and Resources
  13. Quick Start
  14. Conclusions and Future Work
  15. Appendix
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Overview of the KLoM methodology georgiev2024attributetodeletemachineunlearningdatamodel.
  • Figure 1.a: Validation set.