Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI Coders Are Among Us: Rethinking Programming Language Grammar Towards Efficient Code Generation

Zhensu Sun, Xiaoning Du, Zhou Yang, Li Li, David Lo

TL;DR

The paper argues that current human-centric programming grammars impose token-level inefficiencies for AI coders and proposes AI-oriented grammar, instantiated as SimPy for Python. SimPy preserves the same AST as Python while reducing token usage, enabling faster inference, demonstrated across multiple tokenizers with up to 34.7% reductions and up to 10.4% gains for GPT-family tokenizers. The authors explore training strategies showing that models pre-trained on Python and then fine-tuned on SimPy can achieve equal or better performance on code-generation benchmarks like HumanEval, suggesting AI-oriented grammar can be learned without sacrificing accuracy. To translate this into real-world practice, they introduce DualCode, a lightweight conversion framework enabling AI-efficient grammar during inference while preserving human-readable output for collaboration, with converter latencies under 1 ms for typical code sizes. The work points to significant potential for reducing energy and computation costs in AI-assisted coding and invites further automation and language expansion to broaden applicability.

Abstract

Artificial Intelligence (AI) models have emerged as another important audience for programming languages alongside humans and machines, as we enter the era of large language models (LLMs). LLMs can now perform well in coding competitions and even write programs like developers to solve various tasks, including mathematical problems. However, the grammar and layout of current programs are designed to cater the needs of human developers -- with many grammar tokens and formatting tokens being used to make the code easier for humans to read. While this is helpful, such a design adds unnecessary computational work for LLMs, as each token they either use or produce consumes computational resources. To improve inference efficiency and reduce computational costs, we propose the concept of AI-oriented grammar. This aims to represent code in a way that better suits the working mechanism of AI models. Code written with AI-oriented grammar discards formats and uses a minimum number of tokens to convey code semantics effectively. To demonstrate the feasibility of this concept, we explore and implement the first AI-oriented grammar for Python, named SimPy. SimPy is crafted by revising the original Python grammar through a series of heuristic rules. Programs written in SimPy maintain identical AST structures to those in standard Python. This allows for not only execution via a modified AST parser, but also seamless transformation between programs written in Python and SimPy, enabling human developers and LLMs to use Python and SimPy, respectively, when they need to collaborate. In the experiments, compared with Python, SimPy enables a reduction in token usage by 13.5% and 10.4% for CodeLlama and GPT-4, respectively, when completing the same set of code-related tasks. Additionally, these models can maintain or even improve their performance when using SimPy instead of Python for these tasks.

AI Coders Are Among Us: Rethinking Programming Language Grammar Towards Efficient Code Generation

TL;DR

The paper argues that current human-centric programming grammars impose token-level inefficiencies for AI coders and proposes AI-oriented grammar, instantiated as SimPy for Python. SimPy preserves the same AST as Python while reducing token usage, enabling faster inference, demonstrated across multiple tokenizers with up to 34.7% reductions and up to 10.4% gains for GPT-family tokenizers. The authors explore training strategies showing that models pre-trained on Python and then fine-tuned on SimPy can achieve equal or better performance on code-generation benchmarks like HumanEval, suggesting AI-oriented grammar can be learned without sacrificing accuracy. To translate this into real-world practice, they introduce DualCode, a lightweight conversion framework enabling AI-efficient grammar during inference while preserving human-readable output for collaboration, with converter latencies under 1 ms for typical code sizes. The work points to significant potential for reducing energy and computation costs in AI-assisted coding and invites further automation and language expansion to broaden applicability.

Abstract

Artificial Intelligence (AI) models have emerged as another important audience for programming languages alongside humans and machines, as we enter the era of large language models (LLMs). LLMs can now perform well in coding competitions and even write programs like developers to solve various tasks, including mathematical problems. However, the grammar and layout of current programs are designed to cater the needs of human developers -- with many grammar tokens and formatting tokens being used to make the code easier for humans to read. While this is helpful, such a design adds unnecessary computational work for LLMs, as each token they either use or produce consumes computational resources. To improve inference efficiency and reduce computational costs, we propose the concept of AI-oriented grammar. This aims to represent code in a way that better suits the working mechanism of AI models. Code written with AI-oriented grammar discards formats and uses a minimum number of tokens to convey code semantics effectively. To demonstrate the feasibility of this concept, we explore and implement the first AI-oriented grammar for Python, named SimPy. SimPy is crafted by revising the original Python grammar through a series of heuristic rules. Programs written in SimPy maintain identical AST structures to those in standard Python. This allows for not only execution via a modified AST parser, but also seamless transformation between programs written in Python and SimPy, enabling human developers and LLMs to use Python and SimPy, respectively, when they need to collaborate. In the experiments, compared with Python, SimPy enables a reduction in token usage by 13.5% and 10.4% for CodeLlama and GPT-4, respectively, when completing the same set of code-related tasks. Additionally, these models can maintain or even improve their performance when using SimPy instead of Python for these tasks.
Paper Structure (29 sections, 1 theorem, 4 figures, 4 tables)

This paper contains 29 sections, 1 theorem, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Human-centric Grammar Design
  4. AI-Oriented Grammar
  5. Python Code Dataset for Our Study
  6. Token Reduction (RQ1)
  7. An AI-oriented grammar for Python
  8. Design
  9. Unambiguity of SimPy
  10. Semantic equivalence between SimPy and Python
  11. Implementation
  12. Experiments of RQ1
  13. Tokenizers
  14. Results
  15. Model Training (RQ2)
  16. ...and 14 more sections

Key Result

Theorem 1

Python and $\textsc{SimPy }$ are semantically equivalent.

Figures (4)

  • Figure 1: An illustration of how LLMs and human programmers perceive the source code.
  • Figure 2: Illustrations of human-centric design elements in Python.
  • Figure 3: A comparison between Python and SimPy source code, tokenized by GPT-4's tokenizer. Continuous characters with the same background color represent the same token. Notably, there are no line breaks in the SimPy example and we add these line breaks in the figure for our human readers.
  • Figure 4: LEFT: the workflow of the basic usage scenarios of AI-oriented grammar. RIGHT: the workflow of the extended usage scenarios of AI-oriented grammar under DualCode, where the code executor of the AI system in the figure is not necessary.

Theorems & Definitions (1)

  • Theorem 1