State of AI: An Empirical 100 Trillion Token Study with OpenRouter

TL;DR

The paper delivers an empirical panorama of real-world LLM usage following the o1 reasoning model, using a privacy-preserving OpenRouter dataset exceeding $100$ trillion tokens to reveal how developers and users interact with open-source vs proprietary models across tasks, regions, and time. It documents the rise of agentic inference, with multi-step reasoning, tool-calling, and longer interaction sequences becoming commonplace, and shows OSS adoption growing to about one-third of usage while maintaining a dynamic, multi-model market. The study introduces the Cinderella Glass Slipper retention phenomenon, wherein foundational cohorts persist across model generations, highlighting how early workload–model fit can create durable lock-in. It also exposes global diffusion patterns, cost–usage dynamics that defy simple pricing assumptions, and category- and provider-specific usage structures that challenge simple narratives about AI deployment. Together, these findings offer a data-driven foundation for model builders, infrastructure providers, and policy-makers to design, deploy, and evaluate LLM systems aligned with real-world usage and value.

Abstract

The past year has marked a turning point in the evolution and real-world use of large language models (LLMs). With the release of the first widely adopted reasoning model, o1, on December 5th, 2024, the field shifted from single-pass pattern generation to multi-step deliberation inference, accelerating deployment, experimentation, and new classes of applications. As this shift unfolded at a rapid pace, our empirical understanding of how these models have actually been used in practice has lagged behind. In this work, we leverage the OpenRouter platform, which is an AI inference provider across a wide variety of LLMs, to analyze over 100 trillion tokens of real-world LLM interactions across tasks, geographies, and time. In our empirical study, we observe substantial adoption of open-weight models, the outsized popularity of creative roleplay (beyond just the productivity tasks many assume dominate) and coding assistance categories, plus the rise of agentic inference. Furthermore, our retention analysis identifies foundational cohorts: early users whose engagement persists far longer than later cohorts. We term this phenomenon the Cinderella "Glass Slipper" effect. These findings underscore that the way developers and end-users engage with LLMs "in the wild" is complex and multifaceted. We discuss implications for model builders, AI developers, and infrastructure providers, and outline how a data-driven understanding of usage can inform better design and deployment of LLM systems.

Figures (29)

• Figure 1: Open vs closed source models split. Weekly share of total token volume by source type. Lighter blue shades represent open-weight models (China vs Rest-of-World), while dark blue corresponds to proprietary (closed) offerings. Vertical dashed lines mark the release of key open weight models including Llama 3.3 70B, DeepSeek V3, DeepSeek R1, Kimi K2, GPT OSS family, and Qwen 3 Coder.
• Figure 2: Weekly token volume by model type. Stacked bar chart showing total token usage by model category over time. Dark red corresponds to proprietary models (Closed), orange represents Chinese open source models (Chinese OSS), and teal indicates open source models developed outside China (RoW OSS). The chart highlights a gradual increase in OSS token share through 2025, particularly among Chinese OSS models beginning in mid-year.
• Figure 3: Top 15 OSS models over time. Weekly relative token share for leading open source models (stacked area chart). Each colored band represents one model’s contribution to total OSS tokens. The broadening palette over time indicates a more competitive distribution without a single dominant model in recent months.
• Figure 4: OSS model size vs. usage. Weekly share of total OSS token volume served by small, medium, and large models. Percentages are normalized by total OSS usage per week.
• Figure 5: Number of OSS models by size over time. Weekly counts of available open source models, grouped by parameter size category.