Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

TL;DR

This study introduces tokenomics for LLM-based multi-agent software engineering and quantifies how tokens are spent across SDLC stages by mapping ChatDev's internal phases to Design, Coding, Code Completion, Code Review, Testing, and Documentation. Using 30 diverse tasks and a GPT-5 Reasoning model, it finds that Code Review dominates token consumption (about 59.4% on average) and that input tokens drive most usage, indicating a substantial communication tax in agent collaboration. The work provides a data-driven cost map of token distribution, highlights inefficiencies in iterative refinement, and proposes a framework for predicting costs and guiding token-efficient workflow design. These insights pave the way for more sustainable, cost-aware agentic SE workflows and set the stage for cross-system benchmarking and protocol improvements.

Abstract

LLM-based Multi-Agent (LLM-MA) systems are increasingly applied to automate complex software engineering tasks such as requirements engineering, code generation, and testing. However, their operational efficiency and resource consumption remain poorly understood, hindering practical adoption due to unpredictable costs and environmental impact. To address this, we conduct an analysis of token consumption patterns in an LLM-MA system within the Software Development Life Cycle (SDLC), aiming to understand where tokens are consumed across distinct software engineering activities. We analyze execution traces from 30 software development tasks performed by the ChatDev framework using a GPT-5 reasoning model, mapping its internal phases to distinct development stages (Design, Coding, Code Completion, Code Review, Testing, and Documentation) to create a standardized evaluation framework. We then quantify and compare token distribution (input, output, reasoning) across these stages. Our preliminary findings show that the iterative Code Review stage accounts for the majority of token consumption for an average of 59.4% of tokens. Furthermore, we observe that input tokens consistently constitute the largest share of consumption for an average of 53.9%, providing empirical evidence for potentially significant inefficiencies in agentic collaboration. Our results suggest that the primary cost of agentic software engineering lies not in initial code generation but in automated refinement and verification. Our novel methodology can help practitioners predict expenses and optimize workflows, and it directs future research toward developing more token-efficient agent collaboration protocols.

Figures (2)

• Figure 1: Overview of our analysis pipeline.
• Figure 2: Average token usage by phase across all 30 tasks for ChatDev with GPT-5 Reasoning. Error bars represent $\pm1$ standard deviation. Note that not all phases occurred in every task (e.g., Code Completion n=6, Testing n=12).