Generating Risky Samples with Conformity Constraints via Diffusion Models

TL;DR

RiskyDiff introduces a diffusion-based framework that generates risky samples while enforcing category conformity through text and image embeddings. It adds an explicit conformity score, embedding screening, and risky gradient guidance to steer samples toward misclassification without breaking labeled category identity. Across CIFAR-100, ImageNet, PACS, and NICO++, RiskyDiff achieves higher risk, better image quality, and stronger conformity than baselines, while enabling improved generalization when used for augmentation. This approach enhances risk discovery and provides a practical data-centric pathway to bolster model robustness in high-stakes tasks.

Abstract

Although neural networks achieve promising performance in many tasks, they may still fail when encountering some examples and bring about risks to applications. To discover risky samples, previous literature attempts to search for patterns of risky samples within existing datasets or inject perturbation into them. Yet in this way the diversity of risky samples is limited by the coverage of existing datasets. To overcome this limitation, recent works adopt diffusion models to produce new risky samples beyond the coverage of existing datasets. However, these methods struggle in the conformity between generated samples and expected categories, which could introduce label noise and severely limit their effectiveness in applications. To address this issue, we propose RiskyDiff that incorporates the embeddings of both texts and images as implicit constraints of category conformity. We also design a conformity score to further explicitly strengthen the category conformity, as well as introduce the mechanisms of embedding screening and risky gradient guidance to boost the risk of generated samples. Extensive experiments reveal that RiskyDiff greatly outperforms existing methods in terms of the degree of risk, generation quality, and conformity with conditioned categories. We also empirically show the generalization ability of the models can be enhanced by augmenting training data with generated samples of high conformity.

Figures (8)

• Figure 1: Illustration of generating risky samples with conformity constraints. We require generated samples to deceive the target model but conform to the conditioned category, so that generated samples could improve generalization of the target model after being added to training data. Otherwise, there could be severe label noise in generated samples.
• Figure 2: Overall framework of RiskyDiff. The majority of this figure shows a single backward step of the sampling process.
• Figure 3: Risky samples generated by RiskyDiff. Three rows correspond to ImageNet, NICO++, and PACS, each including two categories. The left caption implies ground truth category. The lower caption imply prediction of the target model (ResNet-50).
• Figure 4: Images of cats generated by AdvDiffuser, AdvDiff, and our method with ResNet-50 as the target model on NICO++. It shows that AdvDiffuser and AdvDiff fail to preserve the true characteristics of cats while our method can.
• Figure 5: Ablation study of embedding screening and gradient guidance on NICO++. "Base" is direct generating images using Stable-unCLIP. "Screening" is embedding screening. "Gradient" is gradient guidance.