Censoring chemical data to mitigate dual use risk

TL;DR

The paper addresses dual-use risks in predictive chemistry by proposing data-level mitigation through selective noise applied to sensitive data regions, aiming to preserve openness while reducing misuse. It formally analyzes and empirically tests how perturbing either molecular features (via SMILES replacements) or labels in sensitive regions affects model bias and variance across 1D, MLP, and GCN tasks, including lipophilicity prediction. Key finding: selective feature noise can induce attenuation bias in sensitive regions and decrease predictive accuracy for dangerous compounds, while omission of sensitive data fails to prevent extrapolation in deep learning models; label noise increases variance. The approach offers a model-agnostic path to safer open data sharing, though it requires further refinement to balance protection with accuracy in the non-sensitive region and to handle multiple sensitivity levels.

Abstract

Machine learning models have dual-use potential, potentially serving both beneficial and malicious purposes. The development of open-source models in chemistry has specifically surfaced dual-use concerns around toxicological data and chemical warfare agents. We discuss a chain risk framework identifying three misuse pathways and corresponding mitigation strategies: inference-level, model-level, and data-level. At the data level, we introduce a model-agnostic noising method to increase prediction error in specific desired regions (sensitive regions). Our results show that selective noise induces variance and attenuation bias, whereas simply omitting sensitive data fails to prevent extrapolation. These findings hold for both molecular feature multilayer perceptrons and graph neural networks. Thus, noising molecular structures can enable open sharing of potential dual-use molecular data.

Figures (6)

• Figure 1: A chain risk framework for Dual Use Risks in Predictive Chemistry (DURPC). Starting with an actor with intent to do harm (top left), each barrier must be overcome for the consequences to be realized (bottom right). Three distinct pathways are provided for the step between ideation and generating a novel simplified molecular-input line-entry system (SMILES) structure. Points of intervention and general mitigation strategies are identified by cross-hatches. Our proposed intervention to reduce DURPC (i.e. data-level mitigation) is highlighted in red.
• Figure 2: Summary of strategies for mitigating DURPC, categorized into three levels based on where the strategy is implemented: dataset, model, and inference.
• Figure 3: Fitted cubic curves after applying four different training data perturbations for 100 trials. Left: The gray curve represents ground truth model ($x,y$). Right: Models trained on perturbed data. Four perturbation types are applied in the following order: label noise ($\delta y(y)$) (subplot A), feature noise ($\delta x(y)$) (subplot B), and combined label & feature noise ($\delta y(y)/\delta x(y)$) (subplot C), and selective omission (subplot D). Selective perturbation is applied to either of two data regions: data points with negative labels $y<0$ (orange) and data points with positive labels $y>0$ (green). Inset figure in each subplot shows a corresponding parity plot on raw unseen data.
• Figure 4: Evaluation of MLP Performance with Selective Noise. A. A single instance of MLP parity plot after training on each type of data perturbation. We use $y>2$ here as the threshold for sensitive labels. Noise levels are $n_x=2.0$ for features and $n_y=1.5$ for labels. B. Spearman correlation of $y$ vs. $\hat{f}(x)$ on unseen raw data after training on selectively omitted data, dependent on the amount of sensitive data in the dataset. C. Spearman correlation on unseen raw data after training on selective feature noise. D. Spearman correlation on unseen raw data after training on selective label noise.
• Figure 5: Evaluation of GCN Performance with Selective Noise. A. Spearman correlation of $y$ vs. $\hat{f}(x)$ on unseen raw data after training on selectively omitted data, dependent on the amount of sensitive data in the dataset. B. Spearman correlation on unseen raw data after training on selective feature noise. C. Spearman correlation on unseen raw data after training on selective label noise. *Note: in 90% sensitive data, some data failed to converge after omission, leading to missing trials.