Capturing Semantic Flow of ML-based Systems

TL;DR

This paper proposes semantic flow as a representation that captures internal semantic trajectories of ML-based systems by mapping latent activations and reasoning steps into latent spaces, and summarizing them as Semantic Flow Graphs (SFGs). It formalizes the construction with a unit of analysis $e_i$, latent mapping $embed(i,e_i)$, and aggregation $aggregate(i,s_i,S_i)$, and demonstrates two concrete instances: a CNN for CIFAR-10 and an AutoFL LLM agent, connecting SFGs to traditional control flow via SaCFGs. The work highlights applications in measuring out-of-distribution-ness, debugging, predicting execution results with LIG+GCN, and improving interpretability through domain-tailored latent spaces. By enabling the adaptation of dynamic analysis techniques to ML-based software, semantic flow aims to improve reliability, testing, and explainability of complex AI-enabled systems.

Abstract

ML-based systems are software systems that incorporates machine learning components such as Deep Neural Networks (DNNs) or Large Language Models (LLMs). While such systems enable advanced features such as high performance computer vision, natural language processing, and code generation, their internal behaviour remain largely opaque to traditional dynamic analysis such as testing: existing analysis typically concern only what is observable from the outside, such as input similarity or class label changes. We propose semantic flow, a concept designed to capture the internal behaviour of ML-based system and to provide a platform for traditional dynamic analysis techniques to be adapted to. Semantic flow combines the idea of control flow with internal states taken from executions of ML-based systems, such as activation values of a specific layer in a DNN, or embeddings of LLM responses at a specific inference step of LLM agents. The resulting representation, summarised as semantic flow graphs, can capture internal decisions that are not explicitly represented in the traditional control flow of ML-based systems. We propose the idea of semantic flow, introduce two examples using a DNN and an LLM agent, and finally sketch its properties and how it can be used to adapt existing dynamic analysis techniques for use in ML-based software systems.

Figures (5)

• Figure 1: A Convolutional Neural Network for Image Classification written in PyTorch
• Figure 2: A Classifier Logic expressed in Python
• Figure 3: Outline of AutoFL, an LLM-based FL agent Kang2024ay
• Figure 4: Semantic clusters observed at different layers of a CNN CIFAR-10 Classifier, visualised in 3D with t-SNE. Note that different classes are increasingly separated from each other as they pass layers, gradually forming more visually separated semantic clusters. We argue that such separations can be likened to decision branching in programs.
• Figure 5: LLM Inference Graph by Kim et al. Kim2025aa of FL inferences of AutoFL Kang2024aa for the bug Lang-29 of Defects4J Just:2014aa