Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TDAD: Test-Driven Agentic Development - Reducing Code Regressions in AI Coding Agents via Graph-Based Impact Analysis

Pepe Alonso, Sergio Yovine, Victor A. Braberman

Abstract

AI coding agents can resolve real-world software issues, yet they frequently introduce regressions -- breaking tests that previously passed. Current benchmarks focus almost exclusively on resolution rate, leaving regression behavior under-studied. This paper presents TDAD (Test-Driven Agentic Development), an open-source tool that performs pre-change impact analysis for AI coding agents. TDAD builds a dependency map between source code and tests so that before committing a patch, the agent knows which tests to verify and can self-correct. The map is delivered as a lightweight agent skill -- a static text file the agent queries at runtime. Evaluated on SWE-bench Verified with two open-weight models running on consumer hardware (Qwen3-Coder 30B, 100 instances; Qwen3.5-35B-A3B, 25 instances), TDAD reduced regressions by 70% (6.08% to 1.82%) compared to a vanilla baseline. In contrast, adding TDD procedural instructions without targeted test context increased regressions to 9.94% -- worse than no intervention at all. When deployed as an agent skill with a different model and framework, TDAD improved issue-resolution rate from 24% to 32%, confirming that surfacing contextual information outperforms prescribing procedural workflows. All code, data, and logs are publicly available at https://github.com/pepealonso95/TDAD.

TDAD: Test-Driven Agentic Development - Reducing Code Regressions in AI Coding Agents via Graph-Based Impact Analysis

Abstract

AI coding agents can resolve real-world software issues, yet they frequently introduce regressions -- breaking tests that previously passed. Current benchmarks focus almost exclusively on resolution rate, leaving regression behavior under-studied. This paper presents TDAD (Test-Driven Agentic Development), an open-source tool that performs pre-change impact analysis for AI coding agents. TDAD builds a dependency map between source code and tests so that before committing a patch, the agent knows which tests to verify and can self-correct. The map is delivered as a lightweight agent skill -- a static text file the agent queries at runtime. Evaluated on SWE-bench Verified with two open-weight models running on consumer hardware (Qwen3-Coder 30B, 100 instances; Qwen3.5-35B-A3B, 25 instances), TDAD reduced regressions by 70% (6.08% to 1.82%) compared to a vanilla baseline. In contrast, adding TDD procedural instructions without targeted test context increased regressions to 9.94% -- worse than no intervention at all. When deployed as an agent skill with a different model and framework, TDAD improved issue-resolution rate from 24% to 32%, confirming that surfacing contextual information outperforms prescribing procedural workflows. All code, data, and logs are publicly available at https://github.com/pepealonso95/TDAD.
Paper Structure (41 sections, 1 equation, 3 figures, 7 tables, 1 algorithm)

This paper contains 41 sections, 1 equation, 3 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. AI Coding Agents and Benchmarks
  4. Regression Testing
  5. Graph-Based Code Analysis
  6. TDD and AI Agents
  7. System Design
  8. Architecture Overview
  9. Graph Schema
  10. Indexing Pipeline
  11. AST Parser.
  12. Graph Builder.
  13. Test Linker.
  14. Impact Analysis
  15. Agent Integration: TDAD as a Skill
  16. ...and 26 more sections

Figures (3)

  • Figure 1: Agentic workflow with TDAD. After implementing changes, the agent queries TDAD's test map to identify at-risk tests, verifies them, and self-corrects if regressions are detected. The dashed arrow represents the self-correction loop.
  • Figure 2: TDAD pipeline. Stage 1 indexes the repository into a code--test graph. Stage 2 exports impacted tests as a static test map. The agent uses grep on the test map to find tests to verify.
  • Figure 3: Total P2P failures per approach (Phase 1). GraphRAG+TDD reduces failure count by 72% vs. vanilla and 81% vs. TDD-only.