Automated Extraction of Protocol State Machines from 3GPP Specifications with Domain-Informed Prompts and LLM Ensembles

TL;DR

This work tackles the challenge of automatically extracting protocol state machines from large, evolving 3GPP specifications. It introduces SpecGPT, a three-stage framework that preprocesses documents, uses domain-informed chain-of-thought prompts, and ensembles multiple LLMs to produce reliable FSMs for NAS, NGAP, and PFCP. The approach achieves state-of-the-art performance, including a perfect state extraction and a high F1 score for transitions on NAS, and demonstrates favorable comparisons to Hermes while enabling scalable handling of complex cellular protocols. The results suggest SpecGPT can accelerate protocol verification, testing, and specification enrichment, with practical costs and generalization validated across protocol versions. This work highlights the potential of LLM-based, domain-guided FSM extraction to improve robustness and maintainability in critical telecommunication systems.

Abstract

Mobile telecommunication networks are foundational to global infrastructure and increasingly support critical sectors such as manufacturing, transportation, and healthcare. The security and reliability of these networks are essential, yet depend heavily on accurate modeling of underlying protocols through state machines. While most prior work constructs such models manually from 3GPP specifications, this process is labor-intensive, error-prone, and difficult to maintain due to the complexity and frequent updates of the specifications. Recent efforts using natural language processing have shown promise, but remain limited in handling the scale and intricacy of cellular protocols. In this work, we propose SpecGPT, a novel framework that leverages large language models (LLMs) to automatically extract protocol state machines from 3GPP documents. SpecGPT segments technical specifications into meaningful paragraphs, applies domain-informed prompting with chain-of-thought reasoning, and employs ensemble methods to enhance output reliability. We evaluate SpecGPT on three representative 5G protocols (NAS, NGAP, and PFCP) using manually annotated ground truth, and show that it outperforms existing approaches, demonstrating the effectiveness of LLMs for protocol modeling at scale.

Figures (4)

• Figure 1: 5G Architecture.
• Figure 2: Overview of SpecGPT
• Figure 3: A unified basic prompt model for FSM
• Figure 4: The effectiveness of each model in labeling the tags