Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generating Structured Plan Representation of Procedures with LLMs

Deepeka Garg, Sihan Zeng, Sumitra Ganesh, Leo Ardon

TL;DR

The paper addresses the challenge of inconsistent and unstructured Standard Operating Procedures (SOPs) by introducing SOPStruct, an LLM-driven approach that converts SOPs into a standardized Directed Acyclic Graph (DAG) representation. It uses a three-phase pipeline—segmentation of SOPs into manageable segments, generation of structured subtasks with rich attributes, and a dual evaluation framework combining deterministic PDDL-based validation with non-deterministic LM assessments of initial/goal states and completeness. The method is validated across three cross-domain datasets (Nestful API, RecipeNLG, NL2Process) and demonstrates superior performance over zero-shot, code-style, and BPMN baselines, particularly when segments are used to preserve fine-grained dependencies. The work enables scalable, interpretable, and automatable procedure modeling, paving the way for automated workflow optimization and hybrid human-AI execution, with formal guarantees on graph validity via classical planning.

Abstract

In this paper, we address the challenges of managing Standard Operating Procedures (SOPs), which often suffer from inconsistencies in language, format, and execution, leading to operational inefficiencies. Traditional process modeling demands significant manual effort, domain expertise, and familiarity with complex languages like Business Process Modeling Notation (BPMN), creating barriers for non-techincal users. We introduce SOP Structuring (SOPStruct), a novel approach that leverages Large Language Models (LLMs) to transform SOPs into decision-tree-based structured representations. SOPStruct produces a standardized representation of SOPs across different domains, reduces cognitive load, and improves user comprehension by effectively capturing task dependencies and ensuring sequential integrity. Our approach enables leveraging the structured information to automate workflows as well as empower the human users. By organizing procedures into logical graphs, SOPStruct facilitates backtracking and error correction, offering a scalable solution for process optimization. We employ a novel evaluation framework, combining deterministic methods with the Planning Domain Definition Language (PDDL) to verify graph soundness, and non-deterministic assessment by an LLM to ensure completeness. We empirically validate the robustness of our LLM-based structured SOP representation methodology across SOPs from different domains and varying levels of complexity. Despite the current lack of automation readiness in many organizations, our research highlights the transformative potential of LLMs to streamline process modeling, paving the way for future advancements in automated procedure optimization.

Generating Structured Plan Representation of Procedures with LLMs

TL;DR

The paper addresses the challenge of inconsistent and unstructured Standard Operating Procedures (SOPs) by introducing SOPStruct, an LLM-driven approach that converts SOPs into a standardized Directed Acyclic Graph (DAG) representation. It uses a three-phase pipeline—segmentation of SOPs into manageable segments, generation of structured subtasks with rich attributes, and a dual evaluation framework combining deterministic PDDL-based validation with non-deterministic LM assessments of initial/goal states and completeness. The method is validated across three cross-domain datasets (Nestful API, RecipeNLG, NL2Process) and demonstrates superior performance over zero-shot, code-style, and BPMN baselines, particularly when segments are used to preserve fine-grained dependencies. The work enables scalable, interpretable, and automatable procedure modeling, paving the way for automated workflow optimization and hybrid human-AI execution, with formal guarantees on graph validity via classical planning.

Abstract

In this paper, we address the challenges of managing Standard Operating Procedures (SOPs), which often suffer from inconsistencies in language, format, and execution, leading to operational inefficiencies. Traditional process modeling demands significant manual effort, domain expertise, and familiarity with complex languages like Business Process Modeling Notation (BPMN), creating barriers for non-techincal users. We introduce SOP Structuring (SOPStruct), a novel approach that leverages Large Language Models (LLMs) to transform SOPs into decision-tree-based structured representations. SOPStruct produces a standardized representation of SOPs across different domains, reduces cognitive load, and improves user comprehension by effectively capturing task dependencies and ensuring sequential integrity. Our approach enables leveraging the structured information to automate workflows as well as empower the human users. By organizing procedures into logical graphs, SOPStruct facilitates backtracking and error correction, offering a scalable solution for process optimization. We employ a novel evaluation framework, combining deterministic methods with the Planning Domain Definition Language (PDDL) to verify graph soundness, and non-deterministic assessment by an LLM to ensure completeness. We empirically validate the robustness of our LLM-based structured SOP representation methodology across SOPs from different domains and varying levels of complexity. Despite the current lack of automation readiness in many organizations, our research highlights the transformative potential of LLMs to streamline process modeling, paving the way for future advancements in automated procedure optimization.

Paper Structure

This paper contains 17 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Structured Representation of Procedures
  5. Classical Planning
  6. Methodology
  7. SOP Segmentation Methodology
  8. SOP Structure Generation Methodology
  9. Evaluation Methodology
  10. Classical Planning Approach for Graph Validation
  11. Experiments
  12. Datasets
  13. Baselines
  14. Evaluation
  15. Results and Analysis
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Our Structured Process Generation Methodology: SOPStruct
  • Figure 2: Comparison of plans generated by different methods for a recipe procedure, (a) Code-style, (b) BPMN, (c), Zero-shot, and (d) SOPStruct. For brevity, we only include the starting and final subtasks with descriptions for SOPStruct.