Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Parallelize with OpenMP by Augmented Heterogeneous AST Representation

Le Chen, Quazi Ishtiaque Mahmud, Hung Phan, Nesreen K. Ahmed, Ali Jannesari

TL;DR

This work proposes a novel graph-based learning approach called Graph2Par that utilizes a heterogeneous augmented abstract syntax tree (Augmented-AST) representation for code and achieves the accuracy of parallelizable code region detection with 85% accuracy and outperforms the state-of-the-art token-based machine learning approach.

Abstract

Detecting parallelizable code regions is a challenging task, even for experienced developers. Numerous recent studies have explored the use of machine learning for code analysis and program synthesis, including parallelization, in light of the success of machine learning in natural language processing. However, applying machine learning techniques to parallelism detection presents several challenges, such as the lack of an adequate dataset for training, an effective code representation with rich information, and a suitable machine learning model to learn the latent features of code for diverse analyses. To address these challenges, we propose a novel graph-based learning approach called Graph2Par that utilizes a heterogeneous augmented abstract syntax tree (Augmented-AST) representation for code. The proposed approach primarily focused on loop-level parallelization with OpenMP. Moreover, we create an OMP\_Serial dataset with 18598 parallelizable and 13972 non-parallelizable loops to train the machine learning models. Our results show that our proposed approach achieves the accuracy of parallelizable code region detection with 85\% accuracy and outperforms the state-of-the-art token-based machine learning approach. These results indicate that our approach is competitive with state-of-the-art tools and capable of handling loops with complex structures that other tools may overlook.

Learning to Parallelize with OpenMP by Augmented Heterogeneous AST Representation

TL;DR

This work proposes a novel graph-based learning approach called Graph2Par that utilizes a heterogeneous augmented abstract syntax tree (Augmented-AST) representation for code and achieves the accuracy of parallelizable code region detection with 85% accuracy and outperforms the state-of-the-art token-based machine learning approach.

Abstract

Detecting parallelizable code regions is a challenging task, even for experienced developers. Numerous recent studies have explored the use of machine learning for code analysis and program synthesis, including parallelization, in light of the success of machine learning in natural language processing. However, applying machine learning techniques to parallelism detection presents several challenges, such as the lack of an adequate dataset for training, an effective code representation with rich information, and a suitable machine learning model to learn the latent features of code for diverse analyses. To address these challenges, we propose a novel graph-based learning approach called Graph2Par that utilizes a heterogeneous augmented abstract syntax tree (Augmented-AST) representation for code. The proposed approach primarily focused on loop-level parallelization with OpenMP. Moreover, we create an OMP\_Serial dataset with 18598 parallelizable and 13972 non-parallelizable loops to train the machine learning models. Our results show that our proposed approach achieves the accuracy of parallelizable code region detection with 85\% accuracy and outperforms the state-of-the-art token-based machine learning approach. These results indicate that our approach is competitive with state-of-the-art tools and capable of handling loops with complex structures that other tools may overlook.
Paper Structure (26 sections, 5 equations, 3 figures, 5 tables)

This paper contains 26 sections, 5 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Motivation Examples
  3. Background
  4. Auto-parallelization and Algorithm-based Tools
  5. Machine Learning-based Auto-Parallelization
  6. Code Representations
  7. Heterogeneous Graph Neural Networks (HGNN)
  8. Dataset Selection and Analysis
  9. Open-source code data
  10. Data Processing
  11. Synthetic data
  12. Approach
  13. Code representation
  14. Transforming the Abstract Syntax Tree
  15. Merging the Control Flow Graph
  16. ...and 11 more sections

Figures (3)

  • Figure 1: Proposed methodology. Data collection and generation: our dataset contains data from GitHub crawling, benchmark collection, and synthetic data generation. Data pre-processing: we extracted loops from codes with pre-processing steps, e.g., removing comments and blank lines. We also label the data according to the extracted pragma. Code representation: we generate the AST of each loop data and convert it to our proposed augmented heterogeneous AST. Training and Prediction: we feed our processed data and corresponding labels to the HGT model for 4 different downstream tasks.
  • Figure 2: Category-wise loops missed by renowned parallelization assistant tools. The results are generated using the OMP_Serial dataset introduced in section \ref{['sec:data']}.
  • Figure 3: An example of the proposed heterogeneous augmented AST (Heterogeneous aug-AST) representation of code in Listing 1 is shown. The colored blocks indicate the heterogeneous attributes assigned to the AST nodes. The red and yellow lines represent the control flow graph (CFG) and token representation, respectively.