STELP: Secure Transpilation and Execution of LLM-Generated Programs

TL;DR

STELP addresses the safety risks of running LLM-generated code in production by introducing a secure transpilation-execution pipeline that validates, transforms, and executes code under configurable safety policies. The core architecture comprises an AST Processor, Safe Code Generator and Executor, and a Feedback Generator that guides LLMs to repair unsafe patterns. The authors validate STELP with InjectedHumanEval, a human-validated insecure-code dataset, and demonstrate strong safety (TBR=1.0) and high correctness and competitive latency, outperforming static analyzers like CodeShield. They also show practical use cases in AI-assisted software development and agentic systems, and outline future work to extend to more languages and richer risk-tolerance controls.

Abstract

Rapid evolution of Large Language Models (LLMs) has achieved major advances in reasoning, planning, and function-calling capabilities. Multi-agentic collaborative frameworks using such LLMs place them at the center of solving software development-related tasks such as code generation. However, direct use of LLM generated code in production software development systems is problematic. The code could be unstable or erroneous and contain vulnerabilities such as data poisoning, malicious attacks, and hallucinations that could lead to widespread system malfunctions. This prohibits the adoption of LLM generated code in production AI systems where human code reviews and traditional secure testing tools are impractical or untrustworthy. In this paper, we discuss safety and reliability problems with the execution of LLM generated code and propose a Secure Transpiler and Executor of LLM-Generated Program (STELP), capable of executing LLM-generated code in a controlled and safe manner. STELP secures autonomous production AI systems involving code generation, filling the critical void left by the impracticality or limitations of traditional secure testing methodologies and human oversight. This includes applications such as headless code generation-execution and LLMs that produce executable code snippets as an action plan to be executed in real time. We contribute a human-validated dataset of insecure code snippets and benchmark our approach on publicly available datasets for correctness, safety, and latency. Our results demonstrate that our approach outperforms an existing method by a significant margin, particularly in its ability to safely execute risky code snippets. Warning: This paper contains malicious code snippets that should be run with caution.

Figures (10)

• Figure 1: STELP with an illustrative example - (a) LLM generates an action plan as a python code for a flight booking task. Example shows hallucinated tool call and a malicious resource exhaustion attack with an infinite loop. Unsafe execution of such code leads to major security risks. (b) STELP architecture providing safe execution of LLM generated code configured to risk tolerance levels.
• Figure 2: Comparison between original Python interpreter (top) vs. with STELP incorporated (bottom). STELP intercepts LLM-generated code prior to direct execution, applying the necessary safety controls.
• Figure 3: Stepping through STELP with a code snippet containing basic python operations: 1) The snippet is input to the AST Processor. 2) A parsed AST is produced. 3) STELP transpiles the AST into a safe code with a function dedicated for each AST node type. 4) This code is then executed, with the status, logs, and execution details tracked for regeneration purposes. 4) Details are sent to the feedback generator. 5) A natural language explanation of any notable issues is produced. This is then used by the code-gen LLM to regenerate the snippet with fixes applied. The colored boxes show the corresponding elements between the AST nodes and the functions in the transpiled code.
• Figure 4: Stepping through STELP with a code snippet containing both a benign tool-calling action plan and a harmful line. This figure follows the same flow as Figure \ref{['fig:python_snippet_trans_1']} but emphasizes a few additional points. The configuration fed into STELP is used during the transpilation and execution stage to apply the appropriate safety controls. These include controls on tool usage (permissions, timeouts, retries, etc.) and controls on permitted grammar elements and language builtins. During execution, this snippet will fail as intended, with the execute_Call() function in the transpiled code actually enforcing the controls. Not only will it ensure that the search_wikipedia and add_to_db tools are indeed permitted, it will also catch that __import__ is not in the list of allowed built-ins. The execution error logs are summarized through the feedback generator for the code-gen LLM to use.
• Figure 5: Prompt used for injecting vulnerabilities when creating the InjectedHumanEval benchmark.