SLMFix: Leveraging Small Language Models for Error Fixing with Reinforcement Learning
David Jiahao Fu, Aryan Gupta, Aaron Councilman, David Grove, Yu-Xiong Wang, Vikram Adve
TL;DR
SLMFix addresses the challenge of syntactically and functionally erroneous code produced by large language models, especially for low-resource DSLs, by coupling a frozen general-purpose LLM with an SLM fine-tuned via reinforcement learning to repair static errors. The approach uses a dual reward signal comprised of a static validator and an AST/semantic similarity metric, enabling the SLM to improve both syntactic validity and functional alignment without expensive LLM finetuning. Across Ansible, Bash, and SQL, SLMFix achieves high static pass rates and competitive or superior semantic metrics, with notable gains on Ansible and effectiveness demonstrated for LRPLs. The work also validates AST-based semantic scoring as a practical proxy for functional correctness in the absence of comprehensive test suites and provides a new Ansible dataset to support future DSL-based program generation research.
Abstract
Recent advancements in large language models (LLMs) have shown very impressive capabilities in code generation across many programming languages. However, even state-of-the-art LLMs generate programs that contains syntactic errors and fail to complete the given tasks, especially for low-resource programming languages (LRPLs). In addition, high training cost makes finetuning LLMs unaffordable with constrained computational resources, further undermining the effectiveness of LLMs for code generation. In this work, we propose SLMFix, a novel code generation pipeline that leverages a small language model (SLM) finetuned using reinforcement learning (RL) techniques to fix syntactic errors in LLM-generated programs to improve the quality of LLM-generated programs for domain-specific languages (DSLs). In specific, we applied RL on the SLM for the program repair task using a reward calculated using both a static validator and a static semantic similarity metric. Our experimental results demonstrate the effectiveness and generalizability of our approach across multiple DSLs, achieving more than 95% pass rate on the static validator. Notably, SLMFix brings substantial improvement to the base model and outperforms supervised finetuning approach even for 7B models on a LRPL, showing the potential of our approach as an alternative to traditional finetuning approaches.