Exploiting Missing Data Remediation Strategies using Adversarial Missingness Attacks

TL;DR

This work addresses security risks in learning with missing data by introducing BLAMM, a general bi-level optimization framework that learns adversarial missingness mechanisms $p_{R|X}$ to steer ERM-based models toward malicious objectives. It provides differentiable proxy objectives for common remediation techniques, including complete-case analysis and mean/regression-based imputation, enabling gradient-based attacks. Empirical results on real-world tabular datasets show AM attacks can suppress feature significance and drastically inflate average treatment effects, even under partial data access and with defenses like data valuation sometimes offering limited protection. The findings highlight a systemic vulnerability in standard missing-data pipelines and motivate the development of robust defenses and broader extensions to other remediation methods.

Abstract

Adversarial Missingness (AM) attacks aim to manipulate model fitting by carefully engineering a missing data problem to achieve a specific malicious objective. AM attacks are significantly different from prior data poisoning attacks in that no malicious data inserted and no data is maliciously perturbed. Current AM attacks are feasible only under the assumption that the modeler (victim) uses full-information maximum likelihood methods to handle missingness. This work aims to remedy this limitation of AM attacks; in the approach taken here, the adversary achieves their goal by solving a bi-level optimization problem to engineer the adversarial missingness mechanism, where the lower level problem incorporates a differentiable approximation of the targeted missingness remediation technique. As instantiations of this framework, AM attacks are provided for three popular techniques: (i) complete case analysis, (ii) mean imputation, and (iii) regression-based imputation for general empirical risk minimization (ERM) problems. Experiments on real-world data show that AM attacks are successful with modest levels of missingness (less than 20%). Furthermore, we show on the real-world Twins dataset that AM attacks can manipulate the estimated average treatment effect (ATE) as an instance of the general ERM problems: the adversary succeeds in not only reversing the sign, but also in substantially inflating the ATE values from a true value of -1.61% to a manipulated one as high as 10%. These experimental results hold when the ATE is calculated using multiple regression-based estimators with different architectures, even when the adversary is restricted to modifying only a subset of the training data.

Figures (8)

• Figure 1: The AM threat model: the adversary adds missingness to the initial dataset $\bm{X}$ to obtain $\bar{\bm{X}}$, and the modeler mitigates the missingness to learn a model $\hat{\bm{\theta}}$. Because the adversary carefully engineered the missingness mechanism with a malicious goal in mind, $\hat{\bm{\theta}}$ is steered to minimize the adversarial objective $g(\cdot; \bm{X})$.
• Figure 2: Example of manipulating a logistic regression model for a classification problem. By omitting the x-coordinate of 8.4% of the samples (colored blue), the adversary rotates the optimal decision boundary (left figure) to a horizontal line (right figure) under mean imputation. The classification accuracy decreased by 0.4% but with high confidence the modeler asserts that the x variable has a coefficient close to zero (p value=0.688).
• Figure 3: In AM attacks, the adversary uses the samples in the training set to design a missingness mechanism $\mathrm{p}_{R|X}$, given knowledge of the modeler's technique for mitigating missing data. The adversary then samples a masking matrix from the missingness mechanism, replaces the indicated entries with $\text{NaN}$s, and conveys this poisoned dataset to the modeler. The modeler applies CCA or imputation to the partially observed dataset, then proceeds to learn the model.
• Figure 4: Results when KNN-Shapley (left) and LAVA (right) data valuation defenses are used. The top panels show the average (over 20 trials) $\ell_1$ distances between the coefficient estimated in the poisoned dataset and the adversarial coefficients (blue) and the true coefficients (orange), as a function of the number of samples discarded by the modeler. The bottom panels show the corresponding average $p$-values of the target coefficient on a log-scale.
• Figure 5: Training curve of the neural network for mean imputation attack. Left: Loss, Right: % missingness of the target variable.

Theorems & Definitions (2)

• Proposition 1
• proof