How to unlearn a learned Machine Learning model ?
Seifeddine Achour
TL;DR
Addresses selective forgetting in trained models by unlearning undesired data using the Ethical MSE (EMSE) objective, derived from a Gaussian likelihood that maximizes the wanted-data likelihood while suppressing the unwanted-data contribution. EMSE is demonstrated to convert a model trained on mixed data into one that performs well on wanted data (e.g., $R^2_{\text{wanted}}$ rising from 0.43 to 0.98) while shedding influence from unwanted samples, even under large unwanted-data regimes. The paper introduces Exponential R-squared and Fair R-squared as metrics to quantify representativeness of wanted data and unrepresentativeness of unwanted data, alongside discussion of independence assumptions. It also analyzes practical limitations, including dependence between data categories and sensitivity to the noise parameter $\sigma$, highlighting the method's relevance for data governance and privacy in ML. Overall, EMSE provides a concrete, tunable framework for selective forgetting in regression-like settings.
Abstract
In contemporary times, machine learning (ML) has sparked a remarkable revolution across numerous domains, surpassing even the loftiest of human expectations. However, despite the astounding progress made by ML, the need to regulate its outputs and capabilities has become imperative. A viable approach to address this concern is by exerting control over the data used for its training, more precisely, by unlearning the model from undesired data. In this article, I will present an elegant algorithm for unlearning a machine learning model and visualize its abilities. Additionally, I will elucidate the underlying mathematical theory and establish specific metrics to evaluate both the unlearned model's performance on desired data and its level of ignorance regarding unwanted data.