Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Supersonic: Learning to Generate Source Code Optimizations in C/C++

Zimin Chen, Sen Fang, Martin Monperrus

TL;DR

Supersonic addresses automated source-code optimization at the source level by learning targeted edits from pairs of related programs. It leverages a seq2seq model initialized with CodeBERT to predict diff-based patches, using a three-phase inference pipeline: canonicalization, diff synthesis, and post-processing, and is trained on a large, carefully filtered Codeforces/AIZU/AtCoder dataset with CodeNet-derived pretraining. Compared to OpenAI's GPT-3.5-Turbo and GPT-4 on competitive programming tasks, Supersonic delivers superior optimization performance while being ~600x–3700x smaller, and the diff-based output representation further boosts effectiveness, albeit with some malformed patch rejections that reduce accessible predictions. The work demonstrates the value of compact, patch-like code edits and meticulous data curation for reliable code optimization, with external validity demonstrated via submission to real contest platforms. Overall, Supersonic combines a focused patch-based generation objective with domain-specific pretraining to achieve efficient, scalable source-code optimizations that generalize beyond synthetic benchmarks.

Abstract

Software optimization refines programs for resource efficiency while preserving functionality. Traditionally, it is a process done by developers and compilers. This paper introduces a third option, automated optimization at the source code level. We present Supersonic, a neural approach targeting minor source code modifications for optimization. Using a seq2seq model, Supersonic is trained on C/C++ program pairs ($x_{t}$, $x_{t+1}$), where $x_{t+1}$ is an optimized version of $x_{t}$, and outputs a diff. Supersonic's performance is benchmarked against OpenAI's GPT-3.5-Turbo and GPT-4 on competitive programming tasks. The experiments show that Supersonic not only outperforms both models on the code optimization task but also minimizes the extent of the change with a model more than 600x smaller than GPT-3.5-Turbo and 3700x smaller than GPT-4.

Supersonic: Learning to Generate Source Code Optimizations in C/C++

TL;DR

Supersonic addresses automated source-code optimization at the source level by learning targeted edits from pairs of related programs. It leverages a seq2seq model initialized with CodeBERT to predict diff-based patches, using a three-phase inference pipeline: canonicalization, diff synthesis, and post-processing, and is trained on a large, carefully filtered Codeforces/AIZU/AtCoder dataset with CodeNet-derived pretraining. Compared to OpenAI's GPT-3.5-Turbo and GPT-4 on competitive programming tasks, Supersonic delivers superior optimization performance while being ~600x–3700x smaller, and the diff-based output representation further boosts effectiveness, albeit with some malformed patch rejections that reduce accessible predictions. The work demonstrates the value of compact, patch-like code edits and meticulous data curation for reliable code optimization, with external validity demonstrated via submission to real contest platforms. Overall, Supersonic combines a focused patch-based generation objective with domain-specific pretraining to achieve efficient, scalable source-code optimizations that generalize beyond synthetic benchmarks.

Abstract

Software optimization refines programs for resource efficiency while preserving functionality. Traditionally, it is a process done by developers and compilers. This paper introduces a third option, automated optimization at the source code level. We present Supersonic, a neural approach targeting minor source code modifications for optimization. Using a seq2seq model, Supersonic is trained on C/C++ program pairs (, ), where is an optimized version of , and outputs a diff. Supersonic's performance is benchmarked against OpenAI's GPT-3.5-Turbo and GPT-4 on competitive programming tasks. The experiments show that Supersonic not only outperforms both models on the code optimization task but also minimizes the extent of the change with a model more than 600x smaller than GPT-3.5-Turbo and 3700x smaller than GPT-4.
Paper Structure (37 sections, 4 figures, 4 tables)

This paper contains 37 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Large language models
  4. The State-of-the-art Models at OpenAI
  5. Sequence-to-Sequence learning
  6. Technical Solution: Supersonic
  7. Overview
  8. Canonicalize training source code
  9. Output representation & Diff synthesis
  10. Post-processing synthesized code
  11. Training loop
  12. Training Dataset
  13. Experimental Setup
  14. Research Questions
  15. Test dataset
  16. ...and 22 more sections

Figures (4)

  • Figure 1: The pipeline of Supersonic for code optimization.
  • Figure 2: The comparison between the full program and the diff-based output representation generated by Supersonic. When tokenized, the fully optimized program contains 124 tokens whereas the diff-based output representation contains 71 tokens. The difference will be even bigger if the original solution is longer. A longer output increases the probability of generating an erroneous program because of accumulated mistakes.
  • Figure 3: The string similarity distribution for the similarity between the predicted submission and the original solution. Best viewed in color.
  • Figure 4: %OPT for running time optimization of Supersonic, GPT-3.5-Turbo and GPT-4 if we limit the string similarity at various thresholds between the predicted submission and the original solution.