HyperNet-Adaptation for Diffusion-Based Test Case Generation

TL;DR

HyNeA introduces dataset-free, instance-specific HyperNet adaptation to diffusion models to enable targeted, realistic test-case generation for DL vision systems. By inverting ControlNet’s conditioning via a behavior-driven feedback loop, HyNeA steers generated inputs toward failure modes without relying on curated failure datasets or large retraining. Across classification, binary classification, and object detection tasks, HyNeA delivers higher test-case effectiveness, greater realism, and substantially better efficiency than baselines, with human evaluators endorsing the semantic validity of generated cases. This approach offers a practical, scalable framework for evaluating real-world reliability of DL systems while maintaining input fidelity and controllability.

Abstract

The increasing deployment of deep learning systems requires systematic evaluation of their reliability in real-world scenarios. Traditional gradient-based adversarial attacks introduce small perturbations that rarely correspond to realistic failures and mainly assess robustness rather than functional behavior. Generative test generation methods offer an alternative but are often limited to simple datasets or constrained input domains. Although diffusion models enable high-fidelity image synthesis, their computational cost and limited controllability restrict their applicability to large-scale testing. We present HyNeA, a generative testing method that enables direct and efficient control over diffusion-based generation. HyNeA provides dataset-free controllability through hypernetworks, allowing targeted manipulation of the generative process without relying on architecture-specific conditioning mechanisms or dataset-driven adaptations such as fine-tuning. HyNeA employs a distinct training strategy that supports instance-level tuning to identify failure-inducing test cases without requiring datasets that explicitly contain examples of similar failures. This approach enables the targeted generation of realistic failure cases at substantially lower computational cost than search-based methods. Experimental results show that HyNeA improves controllability and test diversity compared to existing generative test generators and generalizes to domains where failure-labeled training data is unavailable.

Figures (16)

• Figure 1: The diffusion and denoising process in a diffusion model.
• Figure 2: Simplified ControlNet Architecture (HyperNet + LDM), $\ast$ indicates Frozen Parameters, the blue arrows indicate control modulation of blocks in the denoising network. For more details on architecture refer to Zhang et al. zhang2023adding.
• Figure 3: HyNeA's component interaction throughout Optimization. Each component and its respective In- and Outputs are described in the following sections.
• Figure 4: Example Origins and Targets for ImageNet. Predicted Origin and Target class are shown on top of the images.
• Figure 5: Embedding comparison for "pizza" and "teddy bear" across methods.