Assessing LLM code generation quality through path planning tasks

TL;DR

This work assesses the risks of deploying LLM-generated code for safety-critical path planning by evaluating six LLMs on Dijkstra, Voronoi, and RRT across maps of increasing difficulty. The findings show that Dijkstra is often viable, Voronoi is rarely correctly implemented from scratch, and RRT performance varies widely with Grok achieving the best reliability; many failures arise from non-determinism, improper library usage, and prompt sensitivity. The study highlights latent hazards in LLM-generated code for safety-critical systems and argues for rigorous testing and substantial human-in-the-loop oversight, rather than full automation. Overall, LLMs cannot yet replace human engineers for safety-critical software, especially given the need to reason about libraries, runtime, and maintainability.

Abstract

As LLM-generated code grows in popularity, more evaluation is needed to assess the risks of using such tools, especially for safety-critical applications such as path planning. Existing coding benchmarks are insufficient as they do not reflect the context and complexity of safety-critical applications. To this end, we assessed six LLMs' abilities to generate the code for three different path-planning algorithms and tested them on three maps of various difficulties. Our results suggest that LLM-generated code presents serious hazards for path planning applications and should not be applied in safety-critical contexts without rigorous testing.

Figures (4)

• Figure 1: Increasingly difficult path planning maps
• Figure 2: Experiment process diagram
• Figure 3: Hazard analysis
• Figure 4: Performance comparison between GPT-4o, which optimized for the shortest path with correct implementation, DeepSeek, which optimized for the shortest path with incorrect implementation, and Gemini, which did not optimize for the shortest path