Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Small Language Models as Compiler Experts: Auto-Parallelization for Heterogeneous Systems

Prathamesh Devadiga

TL;DR

This work investigates whether small, efficient LLMs can drive auto-parallelization for heterogeneous systems, addressing the limitations of brittle traditional heuristics. By a three-stage pipeline and six prompting strategies, the authors demonstrate that ~1B-parameter models can produce safe, effective parallelizations validated by static analysis and sanitizers. Across 11 kernels and 376 tests, the LLM-driven approach achieves an average speedup of 6.81x and a peak of 43.25x on convolutions, frequently outperforming or matching strong baselines like Polly, GCC, TVM, and Triton. The study highlights scalability and correctness advantages, while acknowledging latency and correctness-room for improvement, and suggests future work on latency reduction, broader hardware support, and multi-language targets.

Abstract

Traditional auto-parallelizing compilers, reliant on rigid heuristics, struggle with the complexity of modern heterogeneous systems. This paper presents a comprehensive evaluation of small (approximately 1B parameter) language-model-driven compiler auto-parallelization. We evaluate three models: gemma3, llama3.2, and qwen2.5, using six reasoning strategies across 11 real-world kernels drawn from scientific computing, graph algorithms, and machine learning. Our system is benchmarked against strong compiler baselines, including LLVM Polly, TVM, and Triton. Across 376 total evaluations, the proposed approach achieves an average speedup of 6.81x and a peak performance of 43.25x on convolution operations. We analyze scalability, verify correctness using multiple sanitizers, and confirm robustness across diverse compilers and hardware platforms. Our results demonstrate that small, efficient language models can serve as powerful reasoning engines for complex compiler optimization tasks.

Small Language Models as Compiler Experts: Auto-Parallelization for Heterogeneous Systems

TL;DR

This work investigates whether small, efficient LLMs can drive auto-parallelization for heterogeneous systems, addressing the limitations of brittle traditional heuristics. By a three-stage pipeline and six prompting strategies, the authors demonstrate that ~1B-parameter models can produce safe, effective parallelizations validated by static analysis and sanitizers. Across 11 kernels and 376 tests, the LLM-driven approach achieves an average speedup of 6.81x and a peak of 43.25x on convolutions, frequently outperforming or matching strong baselines like Polly, GCC, TVM, and Triton. The study highlights scalability and correctness advantages, while acknowledging latency and correctness-room for improvement, and suggests future work on latency reduction, broader hardware support, and multi-language targets.

Abstract

Traditional auto-parallelizing compilers, reliant on rigid heuristics, struggle with the complexity of modern heterogeneous systems. This paper presents a comprehensive evaluation of small (approximately 1B parameter) language-model-driven compiler auto-parallelization. We evaluate three models: gemma3, llama3.2, and qwen2.5, using six reasoning strategies across 11 real-world kernels drawn from scientific computing, graph algorithms, and machine learning. Our system is benchmarked against strong compiler baselines, including LLVM Polly, TVM, and Triton. Across 376 total evaluations, the proposed approach achieves an average speedup of 6.81x and a peak performance of 43.25x on convolution operations. We analyze scalability, verify correctness using multiple sanitizers, and confirm robustness across diverse compilers and hardware platforms. Our results demonstrate that small, efficient language models can serve as powerful reasoning engines for complex compiler optimization tasks.
Paper Structure (17 sections, 3 figures, 11 tables)

This paper contains 17 sections, 3 figures, 11 tables.

Table of Contents

  1. Introduction
  2. System and Methodology
  3. LLM-Guided Dependence Reasoning and Parallelization Decisions
  4. Experimental Evaluation
  5. LLM Model and Prompting Strategy Performance
  6. Advanced Baseline Comparison
  7. Scalability and Correctness Analysis
  8. Scalability
  9. Correctness
  10. Limitations
  11. Conclusion and Future Work
  12. Appendix
  13. Robustness and Portability
  14. Detailed Performance on Real-World Kernels
  15. Sample Code Transformation
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Summary performance analysis of LLM models and prompting strategies.
  • Figure 2: Detailed performance speedup across all evaluated kernels. Each bar represents a specific combination of LLM model and prompting strategy, or a traditional baseline, providing a granular view of performance on a per-kernel basis.
  • Figure 3: Efficiency score distribution for each approach shown as a box plot. This visualization highlights the median efficiency (orange line) as well as the variance and outliers for each configuration, offering deeper insight into the consistency of the parallelization strategies.