Privacy-Preserving Model and Preprocessing Verification for Machine Learning

TL;DR

This work tackles the challenge of verifying preprocessing integrity for ML models trained on sensitive data under privacy constraints. It introduces a framework that combines Local Differential Privacy with interpretable explanations from LIME and SHAP, enabling verification without exposing raw data. Empirical results across three real-world datasets show that ML-based verification excels in binary tasks, while threshold-based methods perform comparably in multi-class settings, with verification accuracy depending on the privacy budget. The approach offers strong privacy guarantees and practical utility for safeguarding data, with future work focusing on improved verification metrics and adaptive privacy strategies.

Abstract

This paper presents a framework for privacy-preserving verification of machine learning models, focusing on models trained on sensitive data. Integrating Local Differential Privacy (LDP) with model explanations from LIME and SHAP, our framework enables robust verification without compromising individual privacy. It addresses two key tasks: binary classification, to verify if a target model was trained correctly by applying the appropriate preprocessing steps, and multi-class classification, to identify specific preprocessing errors. Evaluations on three real-world datasets-Diabetes, Adult, and Student Record-demonstrate that while the ML-based approach is particularly effective in binary tasks, the threshold-based method performs comparably in multi-class tasks. Results indicate that although verification accuracy varies across datasets and noise levels, the framework provides effective detection of preprocessing errors, strong privacy guarantees, and practical applicability for safeguarding sensitive data.

Figures (20)

• Figure 1: LIME explanations from the Adult dataset misc_adult_2.
• Figure 2: SHAP explanations (Shapley values) from the Adult dataset misc_adult_2.
• Figure 3: Overview of the proposed verification framework. Step 1: The researcher applies improper preprocessing on the original dataset $D$ and trains a machine learning model $M_\text{R}$. Step 2: The verifier accesses the differentially-private original dataset $D_\epsilon$ that is provided by the researcher, applies proper preprocessing, and trains a local model $M_\epsilon$. Meanwhile, in Step 3, the verifier simulates various improper preprocessing to $D_\epsilon$ and trains multiple erroneous models $M_{\epsilon}^{'}$. Step 4: In parallel, the verifier sends queries to all trained models from a testing dataset ($D_\text{test}$) and gets their interpretable responses $O$ which contain predictions $\hat{y}$ and explanations ($E$) provided by model explainers (LIME). Step 5: After acquiring response $O_\epsilon$ and $O_{\epsilon}^{'}$ from $M_{\epsilon}$ and $M_{\epsilon}^{'}$, respectively, the verifier builds a machine learning-based classifier ($V_\text{ML}$) and a threshold-based classifier ($V_\text{Th}$). Step 6: The verifier assesses the responses $O_\text{R}$ by checking with the two verifying methods trained in step 5. Ultimately, the verifier determines $M_\text{R}$'s correctness.
• Figure 4: Binary-Class ML-Based Accuracy using LIME explainer on Logistic Regression Model
• Figure 5: Multi-Class ML-Based Accuracy using LIME explainer on Logistic Regression Model