Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RM -RF: Reward Model for Run-Free Unit Test Evaluation

Elena Bruches, Daniil Grebenkin, Mikhail Klementev, Vadim Alperovich, Roman Derunets, Dari Baturova, Georgy Mkrtchyan, Oleg Sedukhin, Ivan Bondarenko, Nikolay Bushkov, Stanislav Moiseev

TL;DR

RM-RF introduces a lightweight run-free reward model that predicts three execution-derived signals—successful compilation, coverage increase, and mutation kill rate—using only source and test code. Trained on a multilingual dataset (Java, Python, Go) and evaluated across zero-shot, full fine-tuning, and LoRA regimes, RM-RF achieves an average F1 of 0.69 on binary targets, closely aligning with execution-based outcomes while dramatically reducing latency and resource use. The approach enables scalable feedback for large-scale test generation and RL-based code optimization, with robust cross-language performance and potential for direct integration into reinforcement learning loops. Overall, RM-RF provides a practical, efficient surrogate for runtime evaluation, facilitating faster iteration and broader exploration in automated test generation workflows.

Abstract

We present RM-RF, a lightweight reward model for run-free evaluation of automatically generated unit tests. Instead of repeatedly compiling and executing candidate tests, RM-RF predicts - from source and test code alone - three execution-derived signals: (1) whether the augmented test suite compiles and runs successfully, (2) whether the generated test cases increase code coverage, and (3) whether the generated test cases improve the mutation kill rate. To train and evaluate RM-RF we assemble a multilingual dataset (Java, Python, Go) of focal files, test files, and candidate test additions labeled by an execution-based pipeline, and we release an associated dataset and methodology for comparative evaluation. We tested multiple model families and tuning regimes (zero-shot, full fine-tuning, and PEFT via LoRA), achieving an average F1 of 0.69 across the three targets. Compared to conventional compile-and-run instruments, RM-RF provides substantially lower latency and infrastructure cost while delivering competitive predictive fidelity, enabling fast, scalable feedback for large-scale test generation and RL-based code optimization.

RM -RF: Reward Model for Run-Free Unit Test Evaluation

TL;DR

RM-RF introduces a lightweight run-free reward model that predicts three execution-derived signals—successful compilation, coverage increase, and mutation kill rate—using only source and test code. Trained on a multilingual dataset (Java, Python, Go) and evaluated across zero-shot, full fine-tuning, and LoRA regimes, RM-RF achieves an average F1 of 0.69 on binary targets, closely aligning with execution-based outcomes while dramatically reducing latency and resource use. The approach enables scalable feedback for large-scale test generation and RL-based code optimization, with robust cross-language performance and potential for direct integration into reinforcement learning loops. Overall, RM-RF provides a practical, efficient surrogate for runtime evaluation, facilitating faster iteration and broader exploration in automated test generation workflows.

Abstract

We present RM-RF, a lightweight reward model for run-free evaluation of automatically generated unit tests. Instead of repeatedly compiling and executing candidate tests, RM-RF predicts - from source and test code alone - three execution-derived signals: (1) whether the augmented test suite compiles and runs successfully, (2) whether the generated test cases increase code coverage, and (3) whether the generated test cases improve the mutation kill rate. To train and evaluate RM-RF we assemble a multilingual dataset (Java, Python, Go) of focal files, test files, and candidate test additions labeled by an execution-based pipeline, and we release an associated dataset and methodology for comparative evaluation. We tested multiple model families and tuning regimes (zero-shot, full fine-tuning, and PEFT via LoRA), achieving an average F1 of 0.69 across the three targets. Compared to conventional compile-and-run instruments, RM-RF provides substantially lower latency and infrastructure cost while delivering competitive predictive fidelity, enabling fast, scalable feedback for large-scale test generation and RL-based code optimization.
Paper Structure (22 sections, 4 equations, 2 figures, 4 tables)

This paper contains 22 sections, 4 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Automated Test Generation
  4. Evaluation of the Test Quality
  5. Reinforcement Learning and Reward Modeling in Code Generation
  6. Learning and Testing via Adversarial RL
  7. Dataset Construction
  8. Data evaluation
  9. Data collection
  10. Project selection
  11. Project execution-based filtering
  12. Content-based filtering
  13. Training dataset structure
  14. Experiments
  15. Training configuration
  16. ...and 7 more sections

Figures (2)

  • Figure 1: Distribution of training dataset samples across programming languages
  • Figure 2: Reward model unit test evaluation instruction (simplified version of prompt for binary targets)