code_transformed: The Influence of Large Language Models on Code

TL;DR

This study assesses whether large language models reshape real-world code by analyzing over 20,000 GitHub repositories linked to arXiv papers from 2020–2025. It presents a comprehensive measurement framework covering naming patterns, cyclomatic and Halstead complexity, maintainability, code similarity, and reasoning-label alignment to quantify LLM-induced style shifts. Key findings indicate LLMs influence human coding style, with increased snake_case usage and longer descriptive names, and that LLM-generated code often exhibits higher maintainability and fewer bugs, particularly when guided by reference solutions. The work demonstrates that LLM-assisted coding leaves detectable traces in real-world repositories and has potential implications for software engineering practice, evaluation, and policy.

Abstract

Coding remains one of the most fundamental modes of interaction between humans and machines. With the rapid advancement of Large Language Models (LLMs), code generation capabilities have begun to significantly reshape programming practices. This development prompts a central question: Have LLMs transformed code style, and how can such transformation be characterized? In this paper, we present a pioneering study that investigates the impact of LLMs on code style, with a focus on naming conventions, complexity, maintainability, and similarity. By analyzing code from over 20,000 GitHub repositories linked to arXiv papers published between 2020 and 2025, we identify measurable trends in the evolution of coding style that align with characteristics of LLM-generated code. For instance, the proportion of snake_case function names in Python code increased from 40.7% in Q1 2023 to 49.8% in Q3 2025. Furthermore, we investigate how LLMs approach algorithmic problems by examining their reasoning processes. Our experimental results may provide the first large-scale empirical evidence that LLMs affect real-world programming style. We release all the experimental dataset and source code at: https://github.com/ignorancex/LLM_code

Figures (18)

• Figure 1: LLMs’ preference for longer variable names and snake_case naming patterns.
• Figure 2: The process of our experiments.
• Figure 3: The four figures in the first row present simulation results derived from Codeforces human-written code, either revised by LLMs or directly generated by LLMs based on problem descriptions. The four figures in the second row show the trends over time in GitHub repositories for Python variable names using snake_case, digit-suffixed function names, and the length of variable names. Model abbreviations: GPT (GPT-4.1), Gemini (Gemini-2.0-flash), DS (DeepSeek-R1-Distill-Qwen-32B), Llama (Llama-4-Maverick), Qw (Qwen3-32B), Gemma (Gemma-3-27B).
• Figure 4: (a)--(d) present the results of code generated by various models on our subset problems evaluated using four metrics. The leftmost bars correspond to human-written code. The evaluated models, from left to right, are Qwen2.5-Coder-32B-Instruct, Qwen3-4B, Qwen3-8B, Qwen3-14B, Qwen3-32B, DeepSeek-V3, DeepSeek-R1, Gemma-3-27B, Llama-3.3-Nemotron-Super-49B-V1, GPT-4o-mini, and Claude-3.5-Sonnet. (e)--(h) illustrate the evolution of Python code on GitHub according to the same metrics.
• Figure 5: Frequency comparison of top 10 algorithms on various models (ANS/REF, Python/C/C++).