OptiML: An End-to-End Framework for Program Synthesis and CUDA Kernel Optimization

TL;DR

OptiML addresses the challenge of turning LLM-generated CUDA kernels into high-performance, correctness-guaranteed code by combining a language-model-driven generation stage with a hardware-feedback-guided search stage. The framework uses OptiML-G to propose strong initial kernels from natural-language prompts and OptiML-X to perform multi-step code transformations via Monte Carlo Tree Search, guided by an LLM-as-a-Judge and profiler-derived proxies. It unifies correctness guarding with performance optimization, and reports interpretable evidence tracing improvements to bottlenecks such as memory traffic and instruction footprint. On a diverse CUDA kernel suite, OptiML consistently outperforms purely LLM-based baselines and optimization-only variants, demonstrating faster convergence with a more interpretable optimization trajectory.

Abstract

Generating high-performance CUDA kernels remains challenging due to the need to navigate a combinatorial space of low-level transformations under noisy and expensive hardware feedback. Although large language models can synthesize functionally correct CUDA code, achieving competitive performance requires systematic exploration and verification of optimization choices. We present OptiML, an end-to-end framework that maps either natural-language intent or input CUDA code to performance-optimized CUDA kernels by formulating kernel optimization as search under verification. OptiML consists of two decoupled stages. When the input is natural language, a Mixture-of-Thoughts generator (OptiML-G) acts as a proposal policy over kernel implementation strategies, producing an initial executable program. A search-based optimizer (OptiML-X) then refines either synthesized or user-provided kernels using Monte Carlo Tree Search over LLM-driven edits, guided by a hardware-aware reward derived from profiler feedback. Each candidate transformation is compiled, verified, and profiled with Nsight Compute, and evaluated by a composite objective that combines runtime with hardware bottleneck proxies and guardrails against regressions. We evaluate OptiML in both synthesis-and-optimize and optimization-only settings on a diverse suite of CUDA kernels. Results show that OptiML consistently discovers verified performance improvements over strong LLM baselines and produces interpretable optimization trajectories grounded in profiler evidence.

Figures (1)

• Figure 1: Overview of OptiML. OptiML-G (left) routes inputs through multiple experts with latent-space collaboration to generate initial kernels. OptiML-X (right) optimizes kernels via MCTS with hardware-aware rewards.