Automated Safety Benchmarking: A Multi-agent Pipeline for LVLMs

TL;DR

This paper tackles the high cost, slow update cycles, and limited discriminative power of existing LVLM safety benchmarks. It introduces VLSafetyBencher, a four-agent pipeline (Data Preprocessing, Generation, Augmentation, Selection) that automates dataset construction and updating for LVLM safety evaluation, enabling a benchmark to be created in about one week at a minimal monetary cost. The authors formalize an optimization-based sampling objective balancing separability, harmfulness, and diversity, and demonstrate that their approach yields benchmarks with significantly stronger discriminative ability (e.g., safety score gap of 70%) than human-constructed baselines. Extensive experiments across 35 LVLMs and ablations validate improved benchmark quality, construction efficiency, and the ability to update static benchmarks, highlighting the practicality and scalability of automated cross-modal safety evaluation. The approach promises a scalable pathway for maintaining reliable, up-to-date safety benchmarks aligned with evolving LVLM capabilities.

Abstract

Large vision-language models (LVLMs) exhibit remarkable capabilities in cross-modal tasks but face significant safety challenges, which undermine their reliability in real-world applications. Efforts have been made to build LVLM safety evaluation benchmarks to uncover their vulnerability. However, existing benchmarks are hindered by their labor-intensive construction process, static complexity, and limited discriminative power. Thus, they may fail to keep pace with rapidly evolving models and emerging risks. To address these limitations, we propose VLSafetyBencher, the first automated system for LVLM safety benchmarking. VLSafetyBencher introduces four collaborative agents: Data Preprocessing, Generation, Augmentation, and Selection agents to construct and select high-quality samples. Experiments validates that VLSafetyBencher can construct high-quality safety benchmarks within one week at a minimal cost. The generated benchmark effectively distinguish safety, with a safety rate disparity of 70% between the most and least safe models.

Figures (9)

• Figure 1: The pipeline of VLSafetyBencher comprising four serialized agents. The Preprocessing agent cleans raw data. The Generation agent constructs cross-modal samples, the Augmentation agent enhances diversity and harmfulness, and finally, the Selection agent employs an iterative optimization algorithm to select optimal samples for composing the benchmark.
• Figure 2: Examples of cross-modal strategies. We present the harmfulness of text and image separately, as well as the intermodal semantic overlapping and misleading.
• Figure 3: Rationality of the proposed cross-modal interaction strategies. (a) Overlap between strategies and image-text harmfulness. (b) Proportions of strategies in the final benchmark.
• Figure 4: Benchmark quality comparison (%) between VLSafetyBencher and existing works.
• Figure 5: Quality comparison (%) of original and updated benchmarks.