Validating LLM-Generated Programs with Metamorphic Prompt Testing
Xiaoyin Wang, Dakai Zhu
TL;DR
The paper tackles the challenge of validating LLM-generated programs without ground-truth solutions. It introduces metamorphic prompt testing, which creates multiple paraphrase prompts, generates a target and paraphrase programs, and uses cross-validation over fuzzed test inputs to detect inconsistencies. On HumanEval with GPT-4, the method detects roughly 75% of erroneous programs with a false positive rate around 8.6%, and gains from using more paraphrase prompts while balancing noise. This approach provides a practical, oracle-free mechanism to improve trust in AI-assisted code generation and can inform post-processing or model refinement strategies.
Abstract
The latest paradigm shift in software development brings in the innovation and automation afforded by Large Language Models (LLMs), showcased by Generative Pre-trained Transformer (GPT), which has shown remarkable capacity to generate code autonomously, significantly reducing the manual effort required for various programming tasks. Although, the potential benefits of LLM-generated code are vast, most notably in efficiency and rapid prototyping, as LLMs become increasingly integrated into the software development lifecycle and hence the supply chain, complex and multifaceted challenges arise as the code generated from these language models carry profound questions on quality and correctness. Research is required to comprehensively explore these critical concerns surrounding LLM-generated code. In this paper, we propose a novel solution called metamorphic prompt testing to address these challenges. Our intuitive observation is that intrinsic consistency always exists among correct code pieces but may not exist among flawed code pieces, so we can detect flaws in the code by detecting inconsistencies. Therefore, we can vary a given prompt to multiple prompts with paraphrasing, and to ask the LLM to acquire multiple versions of generated code, so that we can validate whether the semantic relations still hold in the acquired code through cross-validation. Our evaluation on HumanEval shows that metamorphic prompt testing is able to detect 75 percent of the erroneous programs generated by GPT-4, with a false positive rate of 8.6 percent.