Understanding Network Behaviors through Natural Language Question-Answering

TL;DR

This work targets NL-guided network behavior understanding in large-scale, heterogeneous networks. It introduces NetMind, a framework that (i) chunks long configuration files with a tree-based approach to preserve semantic coherence, (ii) normalizes heterogeneous configurations into a unified fact-graph IR for reliable reasoning, and (iii) uses a hybrid imperative-declarative language executed in a runtime to perform scalable, verifiable network QA. A benchmark pairing real router configurations with expert-verified NL QA pairs demonstrates that NetMind outperforms direct QA and RAG baselines, with notable scalability as network size grows. The results highlight the practical potential of NL-driven network management and point to future gains via reinforcement learning to further enhance reasoning and code generation capabilities.

Abstract

Modern large-scale networks introduce significant complexity in understanding network behaviors, increasing the risk of misconfiguration. Prior work proposed to understand network behaviors by mining network configurations, typically relying on domain-specific languages interfaced with formal models. While effective, they suffer from a steep learning curve and limited flexibility. In contrast, natural language (NL) offers a more accessible and interpretable interface, motivating recent research on NL-guided network behavior understanding. Recent advances in large language models (LLMs) further enhance this direction, leveraging their extensive prior knowledge of network concepts and strong reasoning capabilities. However, three key challenges remain: 1) numerous router devices with lengthy configuration files challenge LLM's long-context understanding ability; 2) heterogeneity across devices and protocols impedes scalability; and 3) complex network topologies and protocols demand advanced reasoning abilities beyond the current capabilities of LLMs. To tackle the above challenges, we propose NetMind, a novel framework for querying networks using NL. Our approach introduces a tree-based configuration chunking strategy to preserve semantic coherence while enabling efficient partitioning. We then construct a unified fact graph as an intermediate representation to normalize vendor-specific configurations. Finally, we design a hybrid imperative-declarative language to reduce the reasoning burden on LLMs and enhance precision. We contribute a benchmark consisting of NL question-answer pairs paired with network configurations. Experiments demonstrate that NetMind achieves accurate and scalable network behavior understanding, outperforming existing baselines.

Figures (7)

• Figure 1: OSPF and BGP configuration examples.
• Figure 2: The overall workflow for answering NL questions about network behaviors. The tree-based chunking algorithm first decomposes lengthy configurations into semantic-coherent paths, which are then mapped to explicit facts. Implicit facts are deduced from rules to construct a fact graph IR. This IR supports to perform declarative and imperative reasoning over the generated hybrid code.
• Figure 3: BNF syntax definition of the hybrid language.
• Figure 4: An exemplified code for answering if the traffic from R1 to R3 prefers path $p1$ over $p2$.
• Figure 5: Comparisons of imperative, declarative and hybrid languages on answering four types of questions.

Theorems & Definitions (2)

• Definition 1: Fact Base
• Definition 2: Rule Set