The Matthew Effect of AI Programming Assistants: A Hidden Bias in Software Evolution
Fei Gu, Zi Liang, Hongzong LI, Jiahao MA
TL;DR
The paper investigates how AI programming assistants influence software ecosystem evolution, revealing a Matthew effect where languages and frameworks with greater popularity receive disproportionately higher AI-assisted coding success. Using a large-scale two-tier benchmark across eight languages and six frameworks, it shows that mainstream ecosystems enjoy significantly higher Pass@1 rates and that gaps widen with problem difficulty. This bias suggests AI tools may reinforce existing dominance, potentially reducing language and framework diversity and slowing innovation. The authors provide reproducible benchmarks and propose future work to counteract ecosystem homogenization through diversity-aware methods and extended evaluation across domains.
Abstract
AI-assisted programming is rapidly reshaping software development, with large language models (LLMs) enabling new paradigms such as vibe coding and agentic coding. While prior works have focused on prompt design and code generation quality, the broader impact of LLM-driven development on the iterative dynamics of software engineering remains underexplored. In this paper, we conduct large-scale experiments on thousands of algorithmic programming tasks and hundreds of framework selection tasks to systematically investigate how AI-assisted programming interacts with the software ecosystem. Our analysis reveals \textbf{a striking Matthew effect: the more popular a programming language or framework, the higher the success rate of LLM-generated code}. The phenomenon suggests that AI systems may reinforce existing popularity hierarchies, accelerating convergence around dominant tools while hindering diversity and innovation. We provide a quantitative characterization of this effect and discuss its implications for the future evolution of programming ecosystems.