Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging AI for Productive and Trustworthy HPC Software: Challenges and Research Directions

Keita Teranishi, Harshitha Menon, William F. Godoy, Prasanna Balaprakash, David Bau, Tal Ben-Nun, Abhinav Bhatele, Franz Franchetti, Michael Franusich, Todd Gamblin, Giorgis Georgakoudis, Tom Goldstein, Arjun Guha, Steven Hahn, Costin Iancu, Zheming Jin, Terry Jones, Tze Meng Low, Het Mankad, Narasinga Rao Miniskar, Mohammad Alaul Haque Monil, Daniel Nichols, Konstantinos Parasyris, Swaroop Pophale, Pedro Valero-Lara, Jeffrey S. Vetter, Samuel Williams, Aaron Young

TL;DR

This paper argues that the post-Moore HPC era, with its high costs and architectural diversity, requires AI-driven approaches to boost productivity and performance portability of HPC software. It assesses fundamental challenges in applying AI—especially LLMs—to HPC, including code generation, ecosystem engineering, ethics, and workforce development, and outlines a roadmap of research directions implemented through two DOE-funded projects, Durban and Ellora. The proposed directions cover fine-tuning specialized LLMs, expanding context windows, leveraging multimodal data, improving source synthesis and transpilation, optimizing dynamic mapping and scheduling, and ensuring trustworthiness and verifiability. The work emphasizes cross-disciplinary collaboration, integration with existing HPC stacks, and the need for ethical guidelines to realize a trustworthy, AI-assisted HPC software ecosystem with practical impact on scientific discovery.

Abstract

We discuss the challenges and propose research directions for using AI to revolutionize the development of high-performance computing (HPC) software. AI technologies, in particular large language models, have transformed every aspect of software development. For its part, HPC software is recognized as a highly specialized scientific field of its own. We discuss the challenges associated with leveraging state-of-the-art AI technologies to develop such a unique and niche class of software and outline our research directions in the two US Department of Energy--funded projects for advancing HPC Software via AI: Ellora and Durban.

Leveraging AI for Productive and Trustworthy HPC Software: Challenges and Research Directions

TL;DR

This paper argues that the post-Moore HPC era, with its high costs and architectural diversity, requires AI-driven approaches to boost productivity and performance portability of HPC software. It assesses fundamental challenges in applying AI—especially LLMs—to HPC, including code generation, ecosystem engineering, ethics, and workforce development, and outlines a roadmap of research directions implemented through two DOE-funded projects, Durban and Ellora. The proposed directions cover fine-tuning specialized LLMs, expanding context windows, leveraging multimodal data, improving source synthesis and transpilation, optimizing dynamic mapping and scheduling, and ensuring trustworthiness and verifiability. The work emphasizes cross-disciplinary collaboration, integration with existing HPC stacks, and the need for ethical guidelines to realize a trustworthy, AI-assisted HPC software ecosystem with practical impact on scientific discovery.

Abstract

We discuss the challenges and propose research directions for using AI to revolutionize the development of high-performance computing (HPC) software. AI technologies, in particular large language models, have transformed every aspect of software development. For its part, HPC software is recognized as a highly specialized scientific field of its own. We discuss the challenges associated with leveraging state-of-the-art AI technologies to develop such a unique and niche class of software and outline our research directions in the two US Department of Energy--funded projects for advancing HPC Software via AI: Ellora and Durban.
Paper Structure (19 sections, 1 figure)

This paper contains 19 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Background on HPC Software
  3. Characteristics:
  4. Requirements for modern software:
  5. AI Challenges for HPC Software
  6. Code generation, translation, and optimization:
  7. Engineering the software ecosystem:
  8. Ethics:
  9. Training and workforce development:
  10. Research Directions
  11. Fine-tuning LLMs:
  12. Enhance context window:
  13. Leverage multimodality:
  14. Enhance existing program source synthesis and compiler optimization methods, auto-tuning, and source-to-source transpilation:
  15. Improve dynamic mapping, scheduling, and data orchestration:
  16. ...and 4 more sections

Figures (1)

  • Figure 1: Research thrusts for Durban and Ellora, which target ASCR's focus on AI Innovations for Scientific Knowledge Synthesis and Software Development.