Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Out-of-Distribution Detection for Neurosymbolic Autonomous Cyber Agents

Ankita Samaddar, Nicholas Potteiger, Xenofon Koutsoukos

TL;DR

The paper addresses the challenge of ensuring trustworthiness for RL-based autonomous cyber-defense agents operating under uncertain runtime conditions. It introduces an OOD Monitoring framework that uses a Probabilistic Neural Network to learn discrete system dynamics and detect out-of-distribution transitions, integrating this safety layer with a neurosymbolic agent built on Evolving Behavior Trees. Through CybORG Scenario 2 experiments with adversarial strategies Meander and B_line, the approach demonstrates effective OOD detection and safe handling of strategy switches, including the need for a GetSafeAction! node to restore to safe states. Overall, the work provides a practical runtime safety mechanism for discrete-state, neurosymbolic cyber defenses and outlines avenues for real-world emulation and online learning of adversaries.

Abstract

Autonomous agents for cyber applications take advantage of modern defense techniques by adopting intelligent agents with conventional and learning-enabled components. These intelligent agents are trained via reinforcement learning (RL) algorithms, and can learn, adapt to, reason about and deploy security rules to defend networked computer systems while maintaining critical operational workflows. However, the knowledge available during training about the state of the operational network and its environment may be limited. The agents should be trustworthy so that they can reliably detect situations they cannot handle, and hand them over to cyber experts. In this work, we develop an out-of-distribution (OOD) Monitoring algorithm that uses a Probabilistic Neural Network (PNN) to detect anomalous or OOD situations of RL-based agents with discrete states and discrete actions. To demonstrate the effectiveness of the proposed approach, we integrate the OOD monitoring algorithm with a neurosymbolic autonomous cyber agent that uses behavior trees with learning-enabled components. We evaluate the proposed approach in a simulated cyber environment under different adversarial strategies. Experimental results over a large number of episodes illustrate the overall efficiency of our proposed approach.

Out-of-Distribution Detection for Neurosymbolic Autonomous Cyber Agents

TL;DR

The paper addresses the challenge of ensuring trustworthiness for RL-based autonomous cyber-defense agents operating under uncertain runtime conditions. It introduces an OOD Monitoring framework that uses a Probabilistic Neural Network to learn discrete system dynamics and detect out-of-distribution transitions, integrating this safety layer with a neurosymbolic agent built on Evolving Behavior Trees. Through CybORG Scenario 2 experiments with adversarial strategies Meander and B_line, the approach demonstrates effective OOD detection and safe handling of strategy switches, including the need for a GetSafeAction! node to restore to safe states. Overall, the work provides a practical runtime safety mechanism for discrete-state, neurosymbolic cyber defenses and outlines avenues for real-world emulation and online learning of adversaries.

Abstract

Autonomous agents for cyber applications take advantage of modern defense techniques by adopting intelligent agents with conventional and learning-enabled components. These intelligent agents are trained via reinforcement learning (RL) algorithms, and can learn, adapt to, reason about and deploy security rules to defend networked computer systems while maintaining critical operational workflows. However, the knowledge available during training about the state of the operational network and its environment may be limited. The agents should be trustworthy so that they can reliably detect situations they cannot handle, and hand them over to cyber experts. In this work, we develop an out-of-distribution (OOD) Monitoring algorithm that uses a Probabilistic Neural Network (PNN) to detect anomalous or OOD situations of RL-based agents with discrete states and discrete actions. To demonstrate the effectiveness of the proposed approach, we integrate the OOD monitoring algorithm with a neurosymbolic autonomous cyber agent that uses behavior trees with learning-enabled components. We evaluate the proposed approach in a simulated cyber environment under different adversarial strategies. Experimental results over a large number of episodes illustrate the overall efficiency of our proposed approach.

Paper Structure

This paper contains 13 sections, 9 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Autonomous Agents for Cyber-Defense
  4. Evolving Behavior Tree based Autonomous Cyber-Defense Agent.
  5. Out-of-distribution Detection
  6. Problem Formulation
  7. OOD Monitoring Algorithm
  8. Integration of OOD Monitoring in the EBT
  9. Experiments and Evaluation
  10. Experimental Setup
  11. Evaluation under different Transition Probability Threshold
  12. Evaluation with EBT
  13. Conclusion and Future Works

Figures (9)

  • Figure 1: Network architecture of CybORG Cage Challenge Scenario 2 with three subnets cage_challenge_2; Subnet 1 with five hosts, Subnet 2 with three enterprise servers and a defender host, Subnet 3 with an operational server and three operational hosts.
  • Figure 2: Optimal Behavior Tree for Autonomous Cyber-Defense potteiger2024design
  • Figure 3: A PNN with input layer of size 1, pattern layer of size $n$ = $N \times \tau$, and output layer of size $m$ where $m$ denotes distinct observed states in $D_{train}$.
  • Figure 4: Updated Behavior Tree for OOD Monitoring; The newly added nodes and connections are marked in red in the updated BT.
  • Figure 5: Software Architecture for OOD Monitoring in autonomous cyber-defense environment.
  • ...and 4 more figures

Theorems & Definitions (2)

  • Definition 1
  • Definition 2