Requirements-driven Slicing of Simulink Models Using LLMs

TL;DR

This paper addresses automated extraction of requirement-based slices from large Simulink models to aid safety-critical CPS certification. It introduces a pipeline that converts Simulink models to text, prompts an LLM to identify blocks needed to satisfy a natural-language requirement, and builds a sound slice from those blocks, analyzing how text granularity and prompting strategy affect accuracy. The study finds that medium-verbosity textual representations combined with chain-of-thought or zero-shot prompting yield the most accurate slices, with $83$ of $180$ slices deemed accurate in the evaluated setup. The work demonstrates the potential of LLM-driven slicing to reduce inspection effort and streamline change-impact analyses, while also highlighting limitations and directions for broader validation across more models and LLMs.

Abstract

Model slicing is a useful technique for identifying a subset of a larger model that is relevant to fulfilling a given requirement. Notable applications of slicing include reducing inspection effort when checking design adequacy to meet requirements of interest and when conducting change impact analysis. In this paper, we present a method based on large language models (LLMs) for extracting model slices from graphical Simulink models. Our approach converts a Simulink model into a textual representation, uses an LLM to identify the necessary Simulink blocks for satisfying a specific requirement, and constructs a sound model slice that incorporates the blocks identified by the LLM. We explore how different levels of granularity (verbosity) in transforming Simulink models into textual representations, as well as the strategy used to prompt the LLM, impact the accuracy of the generated slices. Our preliminary findings suggest that prompts created by textual representations that retain the syntax and semantics of Simulink blocks while omitting visual rendering information of Simulink models yield the most accurate slices. Furthermore, the chain-of-thought and zero-shot prompting strategies result in the largest number of accurate model slices produced by our approach.

Figures (6)

• Figure 1: The Tustin integrator model from Lockheed Martin's Simulink benchmark mavridou2020lockheed. The area outlined in red identifies the blocks that contribute to the satisfaction of requirement $R$: "When $\mathit{reset}$ is True and the Initial Condition ($\mathit{ic}$) is within the Top and Bottom Limits ($\mathit{BL} \leq \mathit{ic} \leq \mathit{TL}$), the Output ($\mathit{yout}$) should match the Initial Condition ($\mathit{ic}$)". The block type is indicated by the label above it, while the number beneath each block represents its SID.
• Figure 2: A slice of the Tustin model (Fig. \ref{['fig:tustin']}) for requirement $R$, obtained using LLM-based slicing. The label above each block specifies its type, and the number below each block denotes its SID.
• Figure 3: An overview of our LLM-based approach: The approach has five inputs and produces, as output, a slice of the input Simulink model for requirement $R$.
• Figure 4: Converting Simulink models to text: (a) An excerpt of the Tustin model in Fig. \ref{['fig:tustin']}, and (b) textual representations of the model excerpt for the high, medium and low verbosity levels.
• Figure 5: Our prompt template; the three placeholders in the template are highlighted in blue. The template is instantiated by replacing the blue text to produce LLM prompts for generating Simulink model slices.