OMP-Engineer: Bridging Syntax Analysis and In-Context Learning for Efficient Automated OpenMP Parallelization

TL;DR

OpenMP parallelization is challenging due to the need for both correct syntactic transformation and performance optimization. The paper introduces OMP-Engineer, a hybrid system that combines syntax-aware AST analysis with in-context learning to guide LLMs in applying OpenMP directives. By generating an AST via ANTLR, detecting non-parallelizable regions and data dependencies, and enriching LLM prompts with targeted examples, the approach achieves accurate and efficient parallel code. Benchmark results on NAS BT, LU, SP demonstrate substantial execution-time reductions compared with Autopar and Par4all, while avoiding inaccuracies observed in ChatGPT-generated code, highlighting a practical path toward reliable automated parallelization.

Abstract

In advancing parallel programming, particularly with OpenMP, the shift towards NLP-based methods marks a significant innovation beyond traditional S2S tools like Autopar and Cetus. These NLP approaches train on extensive datasets of examples to efficiently generate optimized parallel code, streamlining the development process. This method's strength lies in its ability to swiftly produce parallelized code that runs efficiently. However, this reliance on NLP models, without direct code analysis, can introduce inaccuracies, as these models might not fully grasp the nuanced semantics of the code they parallelize. We build OMP-Engineer, which balances the efficiency and scalability of NLP models with the accuracy and reliability of traditional methods, aiming to enhance the performance of automating parallelization while navigating its inherent challenges.