On the interplay of Explainability, Privacy and Predictive Performance with Explanation-assisted Model Extraction
Fatima Ezzeddine, Rinad Akel, Ihab Sbeity, Silvia Giordano, Marc Langheinrich, Omran Ayoub
TL;DR
This paper examines how differential privacy can defend MLaaS systems against model extraction attacks that exploit counterfactual explanations. It investigates DP applied to the target model (DP-SGD), to the counterfactual explainer (DP-GAN), or both, using a KD-based MEA framework and CounterGAN CF generation across two datasets. The study reveals a fundamental privacy–utility trade-off: increasing DP noise strengthens MEA resistance but degrades predictive performance and CF quality, with stronger defense when both DP sources are used. Practically, the work informs design choices for privacy-preserving explanations in explainability-enabled MLaaS, highlighting the need to balance protection against MEA with maintaining usable models and explanations.
Abstract
Machine Learning as a Service (MLaaS) has gained important attraction as a means for deploying powerful predictive models, offering ease of use that enables organizations to leverage advanced analytics without substantial investments in specialized infrastructure or expertise. However, MLaaS platforms must be safeguarded against security and privacy attacks, such as model extraction (MEA) attacks. The increasing integration of explainable AI (XAI) within MLaaS has introduced an additional privacy challenge, as attackers can exploit model explanations particularly counterfactual explanations (CFs) to facilitate MEA. In this paper, we investigate the trade offs among model performance, privacy, and explainability when employing Differential Privacy (DP), a promising technique for mitigating CF facilitated MEA. We evaluate two distinct DP strategies: implemented during the classification model training and at the explainer during CF generation.