Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Automated Kernel Generation in the Era of LLMs

Yang Yu, Peiyu Zang, Chi Hsu Tsai, Haiming Wu, Yixin Shen, Jialing Zhang, Haoyu Wang, Zhiyou Xiao, Jingze Shi, Yuyu Luo, Wentao Zhang, Chunlei Men, Guang Liu, Yonghua Lin

TL;DR

The paper surveys the landscape of LLM-driven kernel generation and optimization, arguing that kernel efficiency is a critical bottleneck for modern AI workloads. It categorizes approaches into supervised fine-tuning and reinforcement learning for kernel synthesis, and analyzes agent-based workflows with learning mechanisms, external memory, hardware profiling, and multi-agent orchestration. It compiles and curates datasets and benchmarks, detailing training corpora and knowledge bases to support learning and evaluation, and outlines data, infrastructure, evaluation, and human–AI collaboration challenges. The work highlights open challenges and practical directions to transition from prototypes to production-grade systems, including scalable data generation, principled agent reasoning, and robust evaluation. An open-source GitHub repository is maintained to catalyze the next generation of automated kernel optimization across heterogeneous hardware.

Abstract

The performance of modern AI systems is fundamentally constrained by the quality of their underlying kernels, which translate high-level algorithmic semantics into low-level hardware operations. Achieving near-optimal kernels requires expert-level understanding of hardware architectures and programming models, making kernel engineering a critical but notoriously time-consuming and non-scalable process. Recent advances in large language models (LLMs) and LLM-based agents have opened new possibilities for automating kernel generation and optimization. LLMs are well-suited to compress expert-level kernel knowledge that is difficult to formalize, while agentic systems further enable scalable optimization by casting kernel development as an iterative, feedback-driven loop. Rapid progress has been made in this area. However, the field remains fragmented, lacking a systematic perspective for LLM-driven kernel generation. This survey addresses this gap by providing a structured overview of existing approaches, spanning LLM-based approaches and agentic optimization workflows, and systematically compiling the datasets and benchmarks that underpin learning and evaluation in this domain. Moreover, key open challenges and future research directions are further outlined, aiming to establish a comprehensive reference for the next generation of automated kernel optimization. To keep track of this field, we maintain an open-source GitHub repository at https://github.com/flagos-ai/awesome-LLM-driven-kernel-generation.

Towards Automated Kernel Generation in the Era of LLMs

TL;DR

The paper surveys the landscape of LLM-driven kernel generation and optimization, arguing that kernel efficiency is a critical bottleneck for modern AI workloads. It categorizes approaches into supervised fine-tuning and reinforcement learning for kernel synthesis, and analyzes agent-based workflows with learning mechanisms, external memory, hardware profiling, and multi-agent orchestration. It compiles and curates datasets and benchmarks, detailing training corpora and knowledge bases to support learning and evaluation, and outlines data, infrastructure, evaluation, and human–AI collaboration challenges. The work highlights open challenges and practical directions to transition from prototypes to production-grade systems, including scalable data generation, principled agent reasoning, and robust evaluation. An open-source GitHub repository is maintained to catalyze the next generation of automated kernel optimization across heterogeneous hardware.

Abstract

The performance of modern AI systems is fundamentally constrained by the quality of their underlying kernels, which translate high-level algorithmic semantics into low-level hardware operations. Achieving near-optimal kernels requires expert-level understanding of hardware architectures and programming models, making kernel engineering a critical but notoriously time-consuming and non-scalable process. Recent advances in large language models (LLMs) and LLM-based agents have opened new possibilities for automating kernel generation and optimization. LLMs are well-suited to compress expert-level kernel knowledge that is difficult to formalize, while agentic systems further enable scalable optimization by casting kernel development as an iterative, feedback-driven loop. Rapid progress has been made in this area. However, the field remains fragmented, lacking a systematic perspective for LLM-driven kernel generation. This survey addresses this gap by providing a structured overview of existing approaches, spanning LLM-based approaches and agentic optimization workflows, and systematically compiling the datasets and benchmarks that underpin learning and evaluation in this domain. Moreover, key open challenges and future research directions are further outlined, aiming to establish a comprehensive reference for the next generation of automated kernel optimization. To keep track of this field, we maintain an open-source GitHub repository at https://github.com/flagos-ai/awesome-LLM-driven-kernel-generation.
Paper Structure (26 sections, 3 equations, 1 figure, 2 tables)

This paper contains 26 sections, 3 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. LLM and LLM-based Autonomous Agents.
  4. Kernel Programming and Code Generation.
  5. LLM for Kernels Generation
  6. Supervised Fine-Tuning
  7. Reinforcement Learning
  8. LLM Agent for Kernels Generation
  9. Learning Mechanisms
  10. External Memory Management
  11. Hardware Profiling Integration
  12. Multi-Agent Orchestration
  13. Datasets for LLM-Based Kernel Generation
  14. Benchmark
  15. Metrics
  16. ...and 11 more sections

Figures (1)

  • Figure 1: Illustration of the growth trend in the field of LLM-driven kernel generation. We organize these research works chronologically and categorically based on their publication dates and the domains they belong to.