Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAFLITE: Fuzzing Autonomous Systems via Large Language Models

Taohong Zhu, Adrians Skapars, Fardeen Mackenzie, Declan Kehoe, William Newton, Suzanne Embury, Youcheng Sun

TL;DR

This work tackles the inefficiency of fuzz testing Autonomous Systems by introducing SaFliTe, an LLM-driven predictor that estimates test-case interestingness with respect to predefined safety objectives. It embeds SaFliTe within a universal AS fuzzing framework, enabling seamless integration with existing fuzzing tools and guiding mutations before execution. Empirical results across several LLMs (GPT-3.5, Mistral-7B, Llama-2-7B) and UAV tools (PGFuzz, DeepHyperion-UAV, CAMBA, TUMB) show substantial improvements in discovering safety violations and policy breaches, including a 93.1% per-round increase in bug-relevant test selection and notable gains in test-case generation. The findings demonstrate the practicality of LLM-guided safety reasoning for fuzzing UAVs and suggest that local, smaller LLMs can be effectively employed with proper prompting, with potential for further refinement through task-specific fine-tuning.

Abstract

Fuzz testing effectively uncovers software vulnerabilities; however, it faces challenges with Autonomous Systems (AS) due to their vast search spaces and complex state spaces, which reflect the unpredictability and complexity of real-world environments. This paper presents a universal framework aimed at improving the efficiency of fuzz testing for AS. At its core is SaFliTe, a predictive component that evaluates whether a test case meets predefined safety criteria. By leveraging the large language model (LLM) with information about the test objective and the AS state, SaFliTe assesses the relevance of each test case. We evaluated SaFliTe by instantiating it with various LLMs, including GPT-3.5, Mistral-7B, and Llama2-7B, and integrating it into four fuzz testing tools: PGFuzz, DeepHyperion-UAV, CAMBA, and TUMB. These tools are designed specifically for testing autonomous drone control systems, such as ArduPilot, PX4, and PX4-Avoidance. The experimental results demonstrate that, compared to PGFuzz, SaFliTe increased the likelihood of selecting operations that triggered bug occurrences in each fuzzing iteration by an average of 93.1\%. Additionally, after integrating SaFliTe, the ability of DeepHyperion-UAV, CAMBA, and TUMB to generate test cases that caused system violations increased by 234.5\%, 33.3\%, and 17.8\%, respectively. The benchmark for this evaluation was sourced from a UAV Testing Competition.

SAFLITE: Fuzzing Autonomous Systems via Large Language Models

TL;DR

This work tackles the inefficiency of fuzz testing Autonomous Systems by introducing SaFliTe, an LLM-driven predictor that estimates test-case interestingness with respect to predefined safety objectives. It embeds SaFliTe within a universal AS fuzzing framework, enabling seamless integration with existing fuzzing tools and guiding mutations before execution. Empirical results across several LLMs (GPT-3.5, Mistral-7B, Llama-2-7B) and UAV tools (PGFuzz, DeepHyperion-UAV, CAMBA, TUMB) show substantial improvements in discovering safety violations and policy breaches, including a 93.1% per-round increase in bug-relevant test selection and notable gains in test-case generation. The findings demonstrate the practicality of LLM-guided safety reasoning for fuzzing UAVs and suggest that local, smaller LLMs can be effectively employed with proper prompting, with potential for further refinement through task-specific fine-tuning.

Abstract

Fuzz testing effectively uncovers software vulnerabilities; however, it faces challenges with Autonomous Systems (AS) due to their vast search spaces and complex state spaces, which reflect the unpredictability and complexity of real-world environments. This paper presents a universal framework aimed at improving the efficiency of fuzz testing for AS. At its core is SaFliTe, a predictive component that evaluates whether a test case meets predefined safety criteria. By leveraging the large language model (LLM) with information about the test objective and the AS state, SaFliTe assesses the relevance of each test case. We evaluated SaFliTe by instantiating it with various LLMs, including GPT-3.5, Mistral-7B, and Llama2-7B, and integrating it into four fuzz testing tools: PGFuzz, DeepHyperion-UAV, CAMBA, and TUMB. These tools are designed specifically for testing autonomous drone control systems, such as ArduPilot, PX4, and PX4-Avoidance. The experimental results demonstrate that, compared to PGFuzz, SaFliTe increased the likelihood of selecting operations that triggered bug occurrences in each fuzzing iteration by an average of 93.1\%. Additionally, after integrating SaFliTe, the ability of DeepHyperion-UAV, CAMBA, and TUMB to generate test cases that caused system violations increased by 234.5\%, 33.3\%, and 17.8\%, respectively. The benchmark for this evaluation was sourced from a UAV Testing Competition.

Paper Structure

This paper contains 24 sections, 4 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Approach
  3. The Universal AS Fuzzing with LLMs
  4. SaFliTe
  5. Definition of Interestingness
  6. Current System State
  7. Mutants
  8. Prompt Template
  9. Evaluation
  10. RQ1: How effective do LLMs predict the interestingness of test cases?
  11. Experimental Setup
  12. Results
  13. RQ2: How does a more specific definition of interestingness impact the effectiveness of SaFliTe?
  14. Experimental Setup
  15. Results
  16. ...and 9 more sections

Figures (4)

  • Figure 1: An Illustrative Workflow of an Autonomous System for UAVs
  • Figure 2: Workflow of the universal AS fuzzing with LLMs
  • Figure 3: A concrete example of SaFliTe applied to a fuzzing tool testing a UAV obstacle avoidance system, PX4-Avoidance. The safety policy of system requires the UAV to maintain a distance of at least 1.5 metres from obstacles. The goal is to find test cases where the UAV either comes within 1.5 metres of an obstacle or collides with one. Each test case represents the number, size, and position of obstacles. In this example, SaFliTe predicts the five test cases, converting the goal into a test condition (a), describing the current state of UAV (b), and summarising the obstacles (c). The LLM analyses and scores each mutant using reasoning, guided by the information in (d).
  • Figure 4: For each bug, SaFliTe categorizes all potential test cases generated by PGFuzz into three groups based on the corresponding Current State and Definition of Interestingness. The figure highlights which category the test case that triggered the bug was assigned to.