Promises, Perils, and (Timely) Heuristics for Mining Coding Agent Activity

TL;DR

The paper investigates the rise of autonomous coding agents and their traceable footprints in GitHub repositories to motivate MSR study. It proposes concrete detection heuristics, supported by a community-driven data repository (agentminingrepo), and synthesizes the promises and perils of mining agent activity. Through sections on agent capabilities, traces, and empirical patterns, the authors outline how files, commits, PRs, issues, and users reveal agent usage, while noting substantial challenges in observability, diversity, and model openness. The work aims to catalyze large-scale, reproducible MSR research into agent-driven software practices and invites community participation to keep heuristics current amid rapid evolution.

Abstract

In 2025, coding agents have seen a very rapid adoption. Coding agents leverage Large Language Models (LLMs) in ways that are markedly different from LLM-based code completion, making their study critical. Moreover, unlike LLM-based completion, coding agents leave visible traces in software repositories, enabling the use of MSR techniques to study their impact on SE practices. This paper documents the promises, perils, and heuristics that we have gathered from studying coding agent activity on GitHub.