Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AIVV: Neuro-Symbolic LLM Agent-Integrated Verification and Validation for Trustworthy Autonomous Systems

Jiyong Kwon, Ujin Jeon, Sooji Lee, Guang Lin

Abstract

Deep learning models excel at detecting anomaly patterns in normal data. However, they do not provide a direct solution for anomaly classification and scalability across diverse control systems, frequently failing to distinguish genuine faults from nuisance faults caused by noise or the control system's large transient response. Consequently, because algorithmic fault validation remains unscalable, full Verification and Validation (V\&V) operations are still managed by Human-in-the-Loop (HITL) analysis, resulting in an unsustainable manual workload. To automate this essential oversight, we propose Agent-Integrated Verification and Validation (AIVV), a hybrid framework that deploys Large Language Models (LLMs) as a deliberative outer loop. Because rigorous system verification strictly depends on accurate validation, AIVV escalates mathematically flagged anomalies to a role-specialized LLM council. The council agents perform collaborative validation by semantically validating nuisance and true failures based on natural-language (NL) requirements to secure a high-fidelity system-verification baseline. Building on this foundation, the council then performs system verification by assessing post-fault responses against NL operational tolerances, ultimately generating actionable V\&V artifacts, such as gain-tuning proposals. Experiments on a time-series simulator for Unmanned Underwater Vehicles (UUVs) demonstrate that AIVV successfully digitizes the HITL V\&V process, overcoming the limitations of rule-based fault classification and offering a scalable blueprint for LLM-mediated oversight in time-series data domains.

AIVV: Neuro-Symbolic LLM Agent-Integrated Verification and Validation for Trustworthy Autonomous Systems

Abstract

Deep learning models excel at detecting anomaly patterns in normal data. However, they do not provide a direct solution for anomaly classification and scalability across diverse control systems, frequently failing to distinguish genuine faults from nuisance faults caused by noise or the control system's large transient response. Consequently, because algorithmic fault validation remains unscalable, full Verification and Validation (V\&V) operations are still managed by Human-in-the-Loop (HITL) analysis, resulting in an unsustainable manual workload. To automate this essential oversight, we propose Agent-Integrated Verification and Validation (AIVV), a hybrid framework that deploys Large Language Models (LLMs) as a deliberative outer loop. Because rigorous system verification strictly depends on accurate validation, AIVV escalates mathematically flagged anomalies to a role-specialized LLM council. The council agents perform collaborative validation by semantically validating nuisance and true failures based on natural-language (NL) requirements to secure a high-fidelity system-verification baseline. Building on this foundation, the council then performs system verification by assessing post-fault responses against NL operational tolerances, ultimately generating actionable V\&V artifacts, such as gain-tuning proposals. Experiments on a time-series simulator for Unmanned Underwater Vehicles (UUVs) demonstrate that AIVV successfully digitizes the HITL V\&V process, overcoming the limitations of rule-based fault classification and offering a scalable blueprint for LLM-mediated oversight in time-series data domains.

Paper Structure

This paper contains 33 sections, 5 equations, 5 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Mathematical Engine Layer
  5. The Multi-Agent System Pipeline
  6. Phase 1: Deterministic Conformal Gating (The Sentry)
  7. Phase 2: Deliberative Adjudication (The Council)
  8. Phase 3: The Adaptation Pipeline (Inspector & Tuner)
  9. V&V Feedback and Corrective-Action Interface
  10. Experimental Setup
  11. Dataset Generation
  12. Failure Mode Generation
  13. Adaptive Engine Configuration
  14. Experiment Results
  15. Validation Performance
  16. ...and 18 more sections

Figures (5)

  • Figure 1: AIVV framework, illustrating the sequential flow of the system.
  • Figure 2: (a) Hovering, (b) Lawnmower Mapping Pattern, and (c) Complex Mission.
  • Figure 2: Failure Validation Rate (FVR) across framework phase.
  • Figure 3: Ablation study comparing three framework stages (rows) across three test scenarios (columns). The mathematical baseline exhibits a high false-positive rate (FPR), which is visibly reduced when the LLM council is introduced. The full AIVV framework integrates the adaptation pipeline to achieve optimal validation.
  • Figure 4: Effect of gain-tuning on REMUS 100 hovering (Dataset 1) yaw response