Configuring Agentic AI Coding Tools: An Exploratory Study
Matthias Galster, Seyedmoein Mohsenimofidi, Jai Lal Lulla, Muhammad Auwal Abubakar, Christoph Treude, Sebastian Baltes
TL;DR
The paper provides the first cross-tool, cross-artifact survey of configuration mechanisms for agentic AI coding tools across Claude Code, GitHub Copilot, Cursor, Gemini, and Codex, examining 2,926 GitHub repositories. It identifies eight configuration mechanisms and shows that Context Files, especially AGENTS.md, dominate the landscape while advanced mechanisms like Skills and Subagents are only shallowly adopted. It also reveals distinct configuration cultures across tools, with Claude Code employing the broadest range of mechanisms. The findings establish an empirical baseline for tracking how configuration strategies evolve and impact agent performance, and they motivate longitudinal and experimental studies on interoperability and deeper automation.
Abstract
Agentic AI coding tools with autonomous capabilities beyond conversational content generation increasingly automate repetitive and time-consuming software development tasks. Developers can configure these tools through versioned repository-level artifacts such as Markdown and JSON files. In this paper, we present a systematic analysis of configuration mechanisms for agentic AI coding tools, covering Claude Code, GitHub Copilot, Cursor, Gemini, and Codex. We identify eight configuration mechanisms and, in an empirical study of 2,926 GitHub repositories, examine whether and how they are adopted. We then explore Context Files, Skills, and Subagents, that is, three mechanisms available across tools, in more detail. Our findings reveal three trends. First, Context Files dominate the configuration landscape and are often the sole mechanism in a repository, with AGENTS$.$md emerging as an interoperable standard across tools. Second, advanced mechanisms such as Skills and Subagents are only shallowly adopted: most repositories define only one or two artifacts, and Skills predominantly rely on static instructions rather than executable workflows. Third, distinct configuration cultures are forming around different tools, with Claude Code users employing the broadest range of mechanisms. These findings establish an empirical baseline for longitudinal and experimental research on how configuration strategies evolve and affect agent performance as agentic AI coding tools mature.