Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

$(ε, δ)$-Differentially Private Partial Least Squares Regression

Ramin Nikzad-Langerodi, Mohit Kumar, Du Nguyen Duy, Mahtab Alghasi

TL;DR

The paper addresses privacy leakage in multivariate calibration by extending Partial Least Squares (PLS) regression with $(ε, δ)$-differential privacy, called edPLS. It injects Gaussian noise into four core PLS outputs—weights, scores, X-loadings, and Y-loadings—calibrating noise via sensitivity bounds to ensure DP while preserving predictive utility. The approach is validated on simulated data and a near-infrared (NIR) corn dataset, showing that stronger privacy (smaller $ε$) effectively blunts privacy attacks, especially when combined with appropriate spectral preprocessing (e.g., Savitzky–Golay). The findings emphasize a practical privacy-utility trade-off for privacy-preserving multivariate calibrations and highlight preprocessing as a key enabler for maintaining utility under strong privacy guarantees, with discussion of extending to federated setups via masking techniques in future work.

Abstract

As data-privacy requirements are becoming increasingly stringent and statistical models based on sensitive data are being deployed and used more routinely, protecting data-privacy becomes pivotal. Partial Least Squares (PLS) regression is the premier tool for building such models in analytical chemistry, yet it does not inherently provide privacy guarantees, leaving sensitive (training) data vulnerable to privacy attacks. To address this gap, we propose an $(ε, δ)$-differentially private PLS (edPLS) algorithm, which integrates well-studied and theoretically motivated Gaussian noise-adding mechanisms into the PLS algorithm to ensure the privacy of the data underlying the model. Our approach involves adding carefully calibrated Gaussian noise to the outputs of four key functions in the PLS algorithm: the weights, scores, $X$-loadings, and $Y$-loadings. The noise variance is determined based on the global sensitivity of each function, ensuring that the privacy loss is controlled according to the $(ε, δ)$-differential privacy framework. Specifically, we derive the sensitivity bounds for each function and use these bounds to calibrate the noise added to the model components. Experimental results demonstrate that edPLS effectively renders privacy attacks, aimed at recovering unique sources of variability in the training data, ineffective. Application of edPLS to the NIR corn benchmark dataset shows that the root mean squared error of prediction (RMSEP) remains competitive even at strong privacy levels (i.e., $ε=1$), given proper pre-processing of the corresponding spectra. These findings highlight the practical utility of edPLS in creating privacy-preserving multivariate calibrations and for the analysis of their privacy-utility trade-offs.

$(ε, δ)$-Differentially Private Partial Least Squares Regression

TL;DR

The paper addresses privacy leakage in multivariate calibration by extending Partial Least Squares (PLS) regression with -differential privacy, called edPLS. It injects Gaussian noise into four core PLS outputs—weights, scores, X-loadings, and Y-loadings—calibrating noise via sensitivity bounds to ensure DP while preserving predictive utility. The approach is validated on simulated data and a near-infrared (NIR) corn dataset, showing that stronger privacy (smaller ) effectively blunts privacy attacks, especially when combined with appropriate spectral preprocessing (e.g., Savitzky–Golay). The findings emphasize a practical privacy-utility trade-off for privacy-preserving multivariate calibrations and highlight preprocessing as a key enabler for maintaining utility under strong privacy guarantees, with discussion of extending to federated setups via masking techniques in future work.

Abstract

As data-privacy requirements are becoming increasingly stringent and statistical models based on sensitive data are being deployed and used more routinely, protecting data-privacy becomes pivotal. Partial Least Squares (PLS) regression is the premier tool for building such models in analytical chemistry, yet it does not inherently provide privacy guarantees, leaving sensitive (training) data vulnerable to privacy attacks. To address this gap, we propose an -differentially private PLS (edPLS) algorithm, which integrates well-studied and theoretically motivated Gaussian noise-adding mechanisms into the PLS algorithm to ensure the privacy of the data underlying the model. Our approach involves adding carefully calibrated Gaussian noise to the outputs of four key functions in the PLS algorithm: the weights, scores, -loadings, and -loadings. The noise variance is determined based on the global sensitivity of each function, ensuring that the privacy loss is controlled according to the -differential privacy framework. Specifically, we derive the sensitivity bounds for each function and use these bounds to calibrate the noise added to the model components. Experimental results demonstrate that edPLS effectively renders privacy attacks, aimed at recovering unique sources of variability in the training data, ineffective. Application of edPLS to the NIR corn benchmark dataset shows that the root mean squared error of prediction (RMSEP) remains competitive even at strong privacy levels (i.e., ), given proper pre-processing of the corresponding spectra. These findings highlight the practical utility of edPLS in creating privacy-preserving multivariate calibrations and for the analysis of their privacy-utility trade-offs.

Paper Structure

This paper contains 19 sections, 28 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Theory
  3. Notation
  4. Differential Privacy
  5. Problem Statement
  6. $(\epsilon, \delta)$-Differentially private PLS
  7. Experimental
  8. Datasets
  9. Simulated Data Set:
  10. NIR of Corn:
  11. Evaluation
  12. Implementation Details:
  13. Simulated Data:
  14. Corn Data:
  15. Results and Discussion
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Motivating example. If the owners of two datasets $( \hbox{\boldmath $\mathrm{X}$} _1, \hbox{\boldmath $\mathrm{y}$} _1)$ and $( \hbox{\boldmath $\mathrm{X}$} _2, \hbox{\boldmath $\mathrm{y}$} _2)$ (left) perform a PLS regression on the concatenated dataset, recovery of the unique sources of variability in the other's $X$-data is trivial (right). This is true even if both parties don't have access to each other's raw data and concatenation/modeling takes place on a remote server.
  • Figure 2: Protection against privacy attacks. The recovery of the unique source of variability (encoded in LV 3) in the other data holder's data according to Eq. \ref{['eq:w_perp']} is shown for decreasing privacy loss $\epsilon$ (at failure probability $\delta=0.01$) using the simulated dataset Figure \ref{['fig:motivating_example']}. The top and bottom plots show weights and loadings inferred by data holder 1, encoding unique sources of variability in data contributed by data holder 2. The dashed lines indicate the position of the unique peak in the other data holder's data (ground truth).
  • Figure 3: Privacy-accuracy trade-off. Regression coefficients (left), the root mean squared error of prediction (RMSEP) divided by the standard deviation of $\hbox{\boldmath $\mathrm{y}$}$ ($\sigma_y$) against privacy loss $\epsilon$ (middle) of edPLS models, and test set predictions of the baseline PLS model with 10 LVs (right) are shown. $R^2P$ indicates the $R^2$ score for the test set.
  • Figure 4: Model selection. The left plot shows the (10-fold) RMSECV vs. the number of LVs for edPLS models with different values of $\epsilon$ ($\delta=0.01$). Errorbars show the mean RMSECV for the corresponding baseline PLS models with the standard error of the mean across 100 experiments. Middle: Regression coefficients for an edPLS model with 2 LVs and $\epsilon=10$. Right: Test set predictions of the best model at $\epsilon=10$.
  • Figure 5: Privacy-preserving spectral pre-processing. Left: Root mean squared error of prediction (RMSEP) on the test set against privacy loss $\epsilon$ based on pre-processed spectra. AirPLS: Adaptive iteratively reweighed penalized least squares baseline correction, MSC: Multiplicative Scatter Correction, SG: Savitzky-Golay (1st derivative) filter. The number of LVs was kept at 8. Middle: The root mean squared error of cross-validation (RMSECV) against the number of LVs for $\epsilon=1$. Right: Test test predictions of the optimal model at $\epsilon=1$ based on SG-preprocessed spectra.