Designing FAIR Workflows at OLCF: Building Scalable and Reusable Ecosystems for HPC Science

TL;DR

The paper argues that high-end HPC centers should cultivate FAIR ecosystems by focusing on FAIRness for individual workflow components rather than entire workflows, enabling cross-disciplinary discovery and reuse. Building on the EOSC-Life model, it outlines a component-centric architecture for OLCF that includes repositories, registries, computing infrastructure, authentication, and metadata standards, tailored to HPC security and embargo needs. It details concrete architectural elements, implementation options, and an adoption strategy that leverages infrastructure upgrades, user interfaces, and incentive structures to drive cultural change. The work highlights the potential for increased reproducibility, reduced duplication of effort, and accelerated scientific discovery through scalable, reusable, and interoperable HPC artifacts.

Abstract

High Performance Computing (HPC) centers provide advanced infrastructure that enables scientific research at extreme scale. These centers operate with hardware configurations, software environments, and security requirements that differ substantially from most users' local systems. As a result, users often develop customized digital artifacts that are tightly coupled to a given HPC center. This practice can lead to significant duplication of effort as multiple users independently create similar solutions to common problems. The FAIR Principles offer a framework to address these challenges. Initially designed to improve data stewardship, the FAIR approach has since been extended to encompass software, workflows, models, and infrastructure. By encouraging the use of rich metadata and community standards, FAIR practices aim to make digital artifacts easier to share and reuse, both within and across scientific domains. Many FAIR initiatives have emerged within individual research communities, often aligned by discipline (e.g. bioinformatics, earth sciences). These communities have made progress in adopting FAIR practices, but their domain-specific nature can lead to silos that limit broader collaboration. Thus, we propose that HPC centers play a more active role in fostering FAIR ecosystems that support research across multiple disciplines. This requires designing infrastructure that enables researchers to discover, share, and reuse computational components more effectively. Here, we build on the architecture of the European Open Science Cloud (EOSC) EOSC-Life FAIR Workflows Collaboratory to propose a model tailored to the needs of HPC. Rather than focusing on entire workflows, we emphasize the importance of making individual workflow components FAIR. This component-based approach better supports the diverse and evolving needs of HPC users while maximizing the long-term value of their work.

Figures (4)

• Figure 1: The EOSC-Life FAIR Workflows Collaboratory wilkinson2025-2.
• Figure 2: Illustrated examples of a FAIR ecosystem in action at an HPC center. Dashed lines (––) show the registry's records tracking components wherever they are. Dotted lines ($\cdots$) show services in an on-prem cloud monitoring the registry and component repository for updates and providing support such as databases to supercomputer jobs. Solid lines (—) show that the supercomputer can consult the registry to locate components and request the service infrastructure to retrieve external artifacts if needed wilkinson2025-3.
• Figure 3: Illustrated example of emerging FAIR ecosystem at OLCF. This is a concrete version of Figure \ref{['fig:diagram']} in which specific infrastructure and services at OLCF are named explicitly. Note that "OLCF Registry" (orange center object) does not exist yet; this is a key difference between the two figures.
• Figure 4: Strategy for Culture Change. Image credit: https://www.cos.io/blog/strategy-for-culture-change.