Bridging the Gap: Empowering Small Models in Reliable OpenACC-based Parallelization via GEPA-Optimized Prompting

TL;DR

The paper addresses the difficulty of reliably generating OpenACC pragmas with LLMs by introducing GEPA-based prompt optimization that evolves simple prompts into highly effective ones for nano-scale models. It develops a semantic-pragma scoring system and a gold-pragma dataset to guide reflective prompt mutations, achieving a robust improvement in compilability and a notable uplift in functional GPU speedups on the PolyBench suite. Key contributions include a two-stage inference process for data management and loop parallelization, and empirical evidence that smaller, cheaper LLMs can match or exceed the pragmas produced by larger models while reducing the risk of incorrect directives. The results demonstrate a scalable, cost-effective pathway to automated directive-based parallelization in HPC workflows, with practical implications for accelerating GPU offloading in scientific computing.

Abstract

OpenACC lowers the barrier to GPU offloading, but writing high-performing pragma remains complex, requiring deep domain expertise in memory hierarchies, data movement, and parallelization strategies. Large Language Models (LLMs) present a promising potential solution for automated parallel code generation, but naive prompting often results in syntactically incorrect directives, uncompilable code, or performance that fails to exceed CPU baselines. We present a systematic prompt optimization approach to enhance OpenACC pragma generation without the prohibitive computational costs associated with model post-training. Leveraging the GEPA (GEnetic-PAreto) framework, we iteratively evolve prompts through a reflective feedback loop. This process utilizes crossover and mutation of instructions, guided by expert-curated gold examples and structured feedback based on clause- and clause parameter-level mismatches between the gold and predicted pragma. In our evaluation on the PolyBench suite, we observe an increase in compilation success rates for programs annotated with OpenACC pragma generated using the optimized prompts compared to those annotated using the simpler initial prompt, particularly for the "nano"-scale models. Specifically, with optimized prompts, the compilation success rate for GPT-4.1 Nano surged from 66.7% to 93.3%, and for GPT-5 Nano improved from 86.7% to 100%, matching or surpassing the capabilities of their significantly larger, more expensive versions. Beyond compilation, the optimized prompts resulted in a 21% increase in the number of programs that achieve functional GPU speedups over CPU baselines. These results demonstrate that prompt optimization effectively unlocks the potential of smaller, cheaper LLMs in writing stable and effective GPU-offloading directives, establishing a cost-effective pathway to automated directive-based parallelization in HPC workflows.

Figures (20)

• Figure 1: Comparison of Gold and Predicted OpenACC Pragma.
• Figure 2: Example of Perfect Semantic Matching.
• Figure 3: Example of Correct Structure, but Incorrect Variables.
• Figure 4: Example of Incorrect Primary Directive.
• Figure 5: symm Benchmark with OpenACC Pragma generated by GPT-4.1 Nano using Initial Prompt