Agent READMEs: An Empirical Study of Context Files for Agentic Coding

TL;DR

This study provides the first large-scale empirical analysis of agent context files (e.g., CLAUDE.md, AGENTS.md, copilot-instructions.md) used to guide autonomous coding agents. It shows that these manifests are long, hard to read, and structurally shallow, with active, incremental maintenance that tracks closely with code evolution. The analysis reveals a strong functional focus (Build & Run, Architecture, Implementation Details) and a notable gap in non-functional requirements like Security and Performance. It demonstrates that automatic classification of these instructions is feasible with a micro-F1 of 0.79, while highlighting limitations for abstract categories and the need for better tooling and governance to manage context debt and ensure safe, reliable agent behavior.

Abstract

Agentic coding tools receive goals written in natural language as input, break them down into specific tasks, and write or execute the actual code with minimal human intervention. Central to this process are agent context files ("READMEs for agents") that provide persistent, project-level instructions. In this paper, we conduct the first large-scale empirical study of 2,303 agent context files from 1,925 repositories to characterize their structure, maintenance, and content. We find that these files are not static documentation but complex, difficult-to-read artifacts that evolve like configuration code, maintained through frequent, small additions. Our content analysis of 16 instruction types shows that developers prioritize functional context, such as build and run commands (62.3%), implementation details (69.9%), and architecture (67.7%). We also identify a significant gap: non-functional requirements like security (14.5%) and performance (14.5%) are rarely specified. These findings indicate that while developers use context files to make agents functional, they provide few guardrails to ensure that agent-written code is secure or performant, highlighting the need for improved tooling and practices.

Figures (6)

• Figure 1: Example of agent context files
• Figure 2: Overview of our methodology.
• Figure 3: Distribution of agent context files (a) size and (b) readability.
• Figure 4: Distribution of agent context files's Headers
• Figure 5: Distribution of agent context files commit activities.