Auto-repair without test cases: How LLMs fix compilation errors in large industrial embedded code
Han Fu, Sigrid Eldh, Kristian Wiklund, Andreas Ermedahl, Philipp Haller, Cyrille Artho
TL;DR
The paper addresses the problem of frequent compilation failures in industrial embedded CI, where test suites are unavailable for non-compilable code. It proposes an LLM-driven automatic repair workflow integrated as a Shadow Job that patches code without disturbing the main CI and validates fixes via a shadow pipeline. Across a year of real-world data and four open-source LLMs, the approach achieves up to 63% CI pass rates and 83% of passing fixes deemed reasonable, with most repairs completed within 8 minutes, significantly speeding up debugging. The work demonstrates the practical viability of leveraging code-focused LLMs for rapid, low-friction repair in industrial CI, with promising implications for deployment, scalability, and future enhancement through fine-tuning and broader evaluation.
Abstract
The co-development of hardware and software in industrial embedded systems frequently leads to compilation errors during continuous integration (CI). Automated repair of such failures is promising, but existing techniques rely on test cases, which are not available for non-compilable code. We employ an automated repair approach for compilation errors driven by large language models (LLMs). Our study encompasses the collection of more than 40000 commits from the product's source code. We assess the performance of an industrial CI system enhanced by four state-of-the-art LLMs, comparing their outcomes with manual corrections provided by human programmers. LLM-equipped CI systems can resolve up to 63 % of the compilation errors in our baseline dataset. Among the fixes associated with successful CI builds, 83 % are deemed reasonable. Moreover, LLMs significantly reduce debugging time, with the majority of successful cases completed within 8 minutes, compared to hours typically required for manual debugging.