Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Code for Machines, Not Just Humans: Quantifying AI-Friendliness with Code Health Metrics

Markus Borg, Nadim Hagatulah, Adam Tornhill, Emma Söderberg

TL;DR

This work defines CodeHealth (CH), a human-oriented maintainability metric, and tests its relation to AI-friendliness in code refactoring by LLMs. Using 5,000 Python solutions from CodeContests and multiple LLMs, the study shows that higher CH ($CH \ge 9$) correlates with lower refactoring break rates and higher semantic preservation, while PPL provides limited predictive value. CH emerges as the strongest single predictor for successful AI-driven refactoring, outperforming SLOC and perplexity across most models, with practical thresholds guiding deployment. The findings support deploying AI-assisted coding more safely by prioritizing or enhancing CH-rich code regions, and they point to broader future work integrating CH with additional structural and multi-agent metrics.

Abstract

We are entering a hybrid era in which human developers and AI coding agents work in the same codebases. While industry practice has long optimized code for human comprehension, it is increasingly important to ensure that LLMs with different capabilities can edit code reliably. In this study, we investigate the concept of ``AI-friendly code'' via LLM-based refactoring on a dataset of 5,000 Python files from competitive programming. We find a meaningful association between CodeHealth, a quality metric calibrated for human comprehension, and semantic preservation after AI refactoring. Our findings confirm that human-friendly code is also more compatible with AI tooling. These results suggest that organizations can use CodeHealth to guide where AI interventions are lower risk and where additional human oversight is warranted. Investing in maintainability not only helps humans; it also prepares for large-scale AI adoption.

Code for Machines, Not Just Humans: Quantifying AI-Friendliness with Code Health Metrics

TL;DR

This work defines CodeHealth (CH), a human-oriented maintainability metric, and tests its relation to AI-friendliness in code refactoring by LLMs. Using 5,000 Python solutions from CodeContests and multiple LLMs, the study shows that higher CH () correlates with lower refactoring break rates and higher semantic preservation, while PPL provides limited predictive value. CH emerges as the strongest single predictor for successful AI-driven refactoring, outperforming SLOC and perplexity across most models, with practical thresholds guiding deployment. The findings support deploying AI-assisted coding more safely by prioritizing or enhancing CH-rich code regions, and they point to broader future work integrating CH with additional structural and multi-agent metrics.

Abstract

We are entering a hybrid era in which human developers and AI coding agents work in the same codebases. While industry practice has long optimized code for human comprehension, it is increasingly important to ensure that LLMs with different capabilities can edit code reliably. In this study, we investigate the concept of ``AI-friendly code'' via LLM-based refactoring on a dataset of 5,000 Python files from competitive programming. We find a meaningful association between CodeHealth, a quality metric calibrated for human comprehension, and semantic preservation after AI refactoring. Our findings confirm that human-friendly code is also more compatible with AI tooling. These results suggest that organizations can use CodeHealth to guide where AI interventions are lower risk and where additional human oversight is warranted. Investing in maintainability not only helps humans; it also prepares for large-scale AI adoption.
Paper Structure (25 sections, 1 equation, 3 figures, 4 tables)

This paper contains 25 sections, 1 equation, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Maintainability and CodeHealth
  4. LLM Confidence and Perplexity
  5. Code Comprehension and Perplexity
  6. AI Refactoring
  7. Method
  8. Dataset Creation
  9. Selection of Large Language Models
  10. RQ$_1$ Perplexity
  11. RQ$_2$ Refactoring Break Rate
  12. RQ$_3$ Predictive Power of CodeHealth
  13. Results
  14. Descriptive Statistics
  15. RQ$_1$ Perplexity
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Distributions of SLOC and CodeHealth.
  • Figure 2: Decision tree for Qwen. Nodes display 1) the split rule (first line, omitted in leaves), 2) the number of samples reaching the node, 3) the class-weighted counts for [fail, pass], and 4) the predicted majority class, also indicated also by color.
  • Figure 3: Refactoring break rate as a function of CodeHealth, binned in intervals of 0.2. Bins with fewer than 10 samples are omitted to reduce noise.