MatchFixAgent: Language-Agnostic Autonomous Repository-Level Code Translation Validation and Repair

TL;DR

Match-Fix-Agent introduces a language-agnostic, multi-agent framework that combines semantic analyses (control-flow, data-flow, IO, library usage, exceptions, and specifications) with LLM-based test generation to validate and repair repository-level code translations across multiple programming languages. The system leverages a three-tier workflow (Semantic Analyzer, Test Generator & Repair Agent, Verdict Agent) to produce a final verdict, accompanying tests, and patches when needed. Evaluation on 2,219 translation pairs across six language pairs shows high verdict coverage and superior repair rates compared with prior tools, while remaining adaptable to different LLMs. The approach is cost-effective, modular, and easily extensible to additional language pairs, addressing key limitations of previous validation and repair techniques.

Abstract

Code translation transforms source code from one programming language (PL) to another. Validating the functional equivalence of translation and repairing, if necessary, are critical steps in code translation. Existing automated validation and repair approaches struggle to generalize to many PLs due to high engineering overhead, and they rely on existing and often inadequate test suites, which results in false claims of equivalence and ineffective translation repair. We develop MatchFixAgent, a large language model (LLM)-based, PL-agnostic framework for equivalence validation and repair of translations. MatchFixAgent features a multi-agent architecture that divides equivalence validation into several sub-tasks to ensure thorough and consistent semantic analysis of the translation. Then it feeds this analysis to test agent to write and execute tests. Upon observing a test failure, the repair agent attempts to fix the translation bug. The final (in)equivalence decision is made by the verdict agent, considering semantic analyses and test execution results. We compare MatchFixAgent's validation and repair results with four repository-level code translation techniques. We use 2,219 translation pairs from their artifacts, which cover 6 PL pairs, and are collected from 24 GitHub projects totaling over 900K lines of code. Our results demonstrate that MatchFixAgent produces (in)equivalence verdicts for 99.2% of translation pairs, with the same equivalence validation result as prior work on 72.8% of them. When MatchFixAgent's result disagrees with prior work, we find that 60.7% of the time MatchFixAgent's result is actually correct. In addition, we show that MatchFixAgent can repair 50.6% of inequivalent translation, compared to prior work's 18.5%. This demonstrates that MatchFixAgent is far more adaptable to many PL pairs than prior work, while producing highly accurate validation results.

Figures (11)

• Figure 1: Illustration of key limitations of existing techniques in validation and repair of repository-level code translation and Match-Fix-Agent addressing them.
• Figure 2: Overview of Match-Fix-Agent.
• Figure 3: CFG and DFP structures extracted by the Semantic Analyzer component in Match-Fix-Agent.
• Figure 4: Prompt structure of Test Generator and Repair Agent.
• Figure 5: Development cost of Match-Fix-Agent compared against existing tools.