Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fixing 7,400 Bugs for 1$: Cheap Crash-Site Program Repair

Han Zheng, Ilia Shumailov, Tianqi Fan, Aiden Hall, Mathias Payer

TL;DR

The work tackles the backlog of vulnerabilities exposed by rapid fuzzing by proposing crash-site repair as a lightweight alternative to root-cause fixes. It introduces WilliamT, a two-component system that uses regex-based crash-site context extraction and template-guided patch generation to minimize reliance on large LLMs and reduce inference costs. Through evaluation on ARVO, the authors show that combining WilliamT with a top APR method can substantially cut token usage and improve fixing efficiency, while also demonstrating scalability to local LLMs on consumer hardware. The approach offers a practical, scalable path for mitigating exploits quickly while giving developers time for deeper root-cause analysis, with potential for open-source deployment.

Abstract

The rapid advancement of bug-finding techniques has led to the discovery of more vulnerabilities than developers can reasonably fix, creating an urgent need for effective Automated Program Repair (APR) methods. However, the complexity of modern bugs often makes precise root cause analysis difficult and unreliable. To address this challenge, we propose crash-site repair to simplify the repair task while still mitigating the risk of exploitation. In addition, we introduce a template-guided patch generation approach that significantly reduces the token cost of Large Language Models (LLMs) while maintaining both efficiency and effectiveness. We implement our prototype system, WILLIAMT, and evaluate it against state-of-the-art APR tools. Our results show that, when combined with the top-performing agent CodeRover-S, WILLIAMT reduces token cost by 45.9% and increases the bug-fixing rate to 73.5% (+29.6%) on ARVO, a ground-truth open source software vulnerabilities benchmark. Furthermore, we demonstrate that WILLIAMT can function effectively even without access to frontier LLMs: even a local model running on a Mac M4 Mini achieves a reasonable repair rate. These findings highlight the broad applicability and scalability of WILLIAMT.

Fixing 7,400 Bugs for 1$: Cheap Crash-Site Program Repair

TL;DR

The work tackles the backlog of vulnerabilities exposed by rapid fuzzing by proposing crash-site repair as a lightweight alternative to root-cause fixes. It introduces WilliamT, a two-component system that uses regex-based crash-site context extraction and template-guided patch generation to minimize reliance on large LLMs and reduce inference costs. Through evaluation on ARVO, the authors show that combining WilliamT with a top APR method can substantially cut token usage and improve fixing efficiency, while also demonstrating scalability to local LLMs on consumer hardware. The approach offers a practical, scalable path for mitigating exploits quickly while giving developers time for deeper root-cause analysis, with potential for open-source deployment.

Abstract

The rapid advancement of bug-finding techniques has led to the discovery of more vulnerabilities than developers can reasonably fix, creating an urgent need for effective Automated Program Repair (APR) methods. However, the complexity of modern bugs often makes precise root cause analysis difficult and unreliable. To address this challenge, we propose crash-site repair to simplify the repair task while still mitigating the risk of exploitation. In addition, we introduce a template-guided patch generation approach that significantly reduces the token cost of Large Language Models (LLMs) while maintaining both efficiency and effectiveness. We implement our prototype system, WILLIAMT, and evaluate it against state-of-the-art APR tools. Our results show that, when combined with the top-performing agent CodeRover-S, WILLIAMT reduces token cost by 45.9% and increases the bug-fixing rate to 73.5% (+29.6%) on ARVO, a ground-truth open source software vulnerabilities benchmark. Furthermore, we demonstrate that WILLIAMT can function effectively even without access to frontier LLMs: even a local model running on a Mac M4 Mini achieves a reasonable repair rate. These findings highlight the broad applicability and scalability of WILLIAMT.
Paper Structure (15 sections, 15 figures, 1 table)

This paper contains 15 sections, 15 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. LLM-Based Program Repairing
  4. Memory Corruption Vulnerabilities and OSS-Fuzz Benchmarks
  5. Primer on Bug Fixing
  6. WilliamT Design
  7. Evaluation
  8. RQ1: Does WilliamT Outperforms Other SoTAs?
  9. RQ2: What kind of bugs does WilliamT fix?
  10. RQ3: Does WilliamT scale on other LLMs?
  11. Discussion
  12. Conclusion
  13. The Plausible Fix and Actual Fix
  14. The Vulnerability Fixing Template
  15. The Regex and Prompt Preparation

Figures (15)

  • Figure 1: Workflow of Current LLM-based Program Repairing Agents vs WilliamT. stands for the LLM components while means non-LLM modules.
  • Figure 2: Example Memory Corruption Vulnerabilties.
  • Figure 3: Root Cause and Crash Site of Bug 66992.
  • Figure 4: The fix performance of WilliamT (with GPT-4o) and other SoTAs zhang2024fixingzhou2024outxia2024automated. Multiple means the CodeRover-S takes multiple attempts to find plausible fix.
  • Figure 5: The WilliamT performance with different LLMs. ds: DeepSeek, gm: Gemma3.
  • ...and 10 more figures