Strategies to Measure Energy Consumption Using RAPL During Workflow Execution on Commodity Clusters

TL;DR

This work addresses the challenge of measuring energy consumption for scientific workflows on commodity clusters using Intel RAPL within a Kubernetes+Nextflow setting. It evaluates four practical approaches (shell-script wrapper, workflow-integrated management, Nextflow plugin, and Prometheus) to capture per-task and per-workflow energy, highlighting that shell-script and plugin-based methods provide strong practicality and coverage, while Prometheus can misestimate for short tasks due to polling intervals. The study demonstrates that per-task energy accounting is straightforward when tasks execute sequentially but requires heuristics for concurrent tasks, and it shows that conventional shell-based measurement best achieves full coverage, with plugin and Prometheus offering viable alternatives under different constraints. These findings lay groundwork for energy-aware workflow optimization on shared clusters and motivate future work on concurrency-aware energy attribution and predictive energy models.

Abstract

In science, problems in many fields can be solved by processing datasets using a series of computationally expensive algorithms, sometimes referred to as workflows. Traditionally, the configurations of these workflows are optimized to achieve a short runtime for the given task and dataset on a given (often distributed) infrastructure. However, recently more attention has been drawn to energy-efficient computing, due to the negative impact of energy-inefficient computing on the environment and energy costs. To be able to assess the energy-efficiency of a given workflow configuration, reliable and accurate methods to measure the energy consumption of a system are required. One approach is the usage of built-in hardware energy counters, such as Intel RAPL. Unfortunately, effectively using RAPL for energy measurement within a workflow on a managed cluster with the typical deep software infrastructure stack can be difficult, for instance because of limited privileges and the need for communication between nodes. In this paper, we describe three ways to implement RAPL energy measurement on a Kubernetes cluster while executing scientific workflows utilizing the Nextflow workflow engine. We compare them by utilizing a set of eight criteria that should be fulfilled for accurate measurement, such as the ability to react to workflow faults, portability, and added overhead. We highlight advantages and drawbacks of each method and discuss challenges and pitfalls, as well as ways to avoid them. We also empirically evaluate all methods, and find that approaches using a shell script and a Nextflow plugin are both effective and easy to implement. Additionally, we find that measuring the energy consumption of a single task is straight forward when only one task runs at a time, but concurrent task executions on the same node require approximating per-task energy usage using metrics such as CPU utilization.

Figures (10)

• Figure 1: The installation locations for external and internal physical power meters lin_taxonomy_2021.
• Figure 2: An overview of the system components covered by each RAPL domain khan_rapl_2018.
• Figure 3: A breakdown of the typical power consumption of a physical server barroso_datacenter_2018. The figure assumes two-socket x86 servers and 12 DIMMs per server, and an average utilization of 80%.
• Figure 4: The communication between the machines when controlling the RAPL measurement as part of the workflow. Each arrow represents a communication between two components of the cluster, or between a component of the cluster and the local machine of the user running the workflow. Solid arrows represent communication directly related to the process of measuring RAPL values, while dashed arrows show communication for workflow execution and extraction of results. The numerated annotations describe the actions that are part of the measurement process in ascending order, starting with 1. as the first action and ending with 7. as the last action of the measurement process. Note that the figure shows two worker nodes and one separate command pod. In practice, the number of used worker nodes depends on the cluster configuration and on the executed workflow. The command pod can be executed on a separate node of the cluster or on one of the worker nodes. During the process of energy measurement, it can be treated like any other pod that is part of the executed workflow.
• Figure 5: The communication between the machines when controlling the RAPL measurement from the machine of the user using a shell-script. Each arrow represents a communication between two components of the cluster, or between a component of the cluster and the local machine of the user running the workflow. Solid arrows represent communication directly related to the process of measuring RAPL values, while dashed arrows show communication for workflow execution and extraction of results. The numerated annotations describe the actions that are part of the measurement process in ascending order, starting with 1. as the first action and ending with 6. as the last action of the measurement process. Note that the figure shows two worker nodes and one separate command pod. In practice, the number of used worker nodes depends on the cluster configuration and on the executed workflow. The command pod can be executed on a separate node of the cluster or on one of the worker nodes. During the process of energy measurement, it can be treated like any other pod that is part of the executed workflow.