ASTRAL: Automated Safety Testing of Large Language Models

TL;DR

ASTRAL tackles the challenge of safely testing large language models by automating the generation and evaluation of unsafe prompts. It introduces a black‑box safety coverage framework spanning safety categories, writing styles, and persuasion techniques, and leverages Retrieval Augmented Generation with few‑shot prompts and live web browsing to produce up‑to‑date inputs. An automated test oracle based on another LLM classifies outputs as safe, unsafe, or unknown, enabling fully automated testing. Empirical results across nine LLMs show that GPT‑3.5 is the most reliable oracle and that ASTRAL substantially increases the detection of unsafe behaviors compared to static baselines, with ablations highlighting the value of few‑shot prompts and web browsing. The work advances practical, scalable safety testing for production‑scale LLMs and suggests directions for richer prompt design and regulatory alignment.

Abstract

Large Language Models (LLMs) have recently gained attention due to their ability to understand and generate sophisticated human-like content. However, ensuring their safety is paramount as they might provide harmful and unsafe responses. Existing LLM testing frameworks address various safety-related concerns (e.g., drugs, terrorism, animal abuse) but often face challenges due to unbalanced and obsolete datasets. In this paper, we present ASTRAL, a tool that automates the generation and execution of test cases (i.e., prompts) for testing the safety of LLMs. First, we introduce a novel black-box coverage criterion to generate balanced and diverse unsafe test inputs across a diverse set of safety categories as well as linguistic writing characteristics (i.e., different style and persuasive writing techniques). Second, we propose an LLM-based approach that leverages Retrieval Augmented Generation (RAG), few-shot prompting strategies and web browsing to generate up-to-date test inputs. Lastly, similar to current LLM test automation techniques, we leverage LLMs as test oracles to distinguish between safe and unsafe test outputs, allowing a fully automated testing approach. We conduct an extensive evaluation on well-known LLMs, revealing the following key findings: i) GPT3.5 outperforms other LLMs when acting as the test oracle, accurately detecting unsafe responses, and even surpassing more recent LLMs (e.g., GPT-4), as well as LLMs that are specifically tailored to detect unsafe LLM outputs (e.g., LlamaGuard); ii) the results confirm that our approach can uncover nearly twice as many unsafe LLM behaviors with the same number of test inputs compared to currently used static datasets; and iii) our black-box coverage criterion combined with web browsing can effectively guide the LLM on generating up-to-date unsafe test inputs, significantly increasing the number of unsafe LLM behaviors.

Figures (7)

• Figure 1: Few-shots examples based test generator user message prompt. Style: slang, Persuasion: expert endorsement
• Figure 2: Tavily search-based test input for controversial topics and politics, using slang and evidence-based persuasion
• Figure 3: System message prompt for the test oracle
• Figure 4: User message prompt for the test oracle
• Figure 5: $RQ_1$ and $RQ_2$: Comparison of unsafe results between the baseline, RAG, RAG + Few-shots, RAG + Few-shots + Tavily search test generators across different execution models.