Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RCOMPSs: A Scalable Runtime System for R Code Execution on Manycore Systems

Xiran Zhang, Javier Conejero, Sameh Abdulah, Jorge Ejarque, Ying Sun, Rosa M. Badia, David E. Keyes, Marc G. Genton

TL;DR

RCOMPSs tackles the scalability gap in R by introducing a COMPSs-based runtime binding that lets R developers write sequential code while the runtime exposes tasks, builds a dynamic DAG, and orchestrates distributed execution. The approach yields automatic data movement, fault tolerance, and back-end agnosticism, demonstrated through KNN, K-means, and linear regression on two HPC systems with strong and weak scalability up to 128 cores per node and 32 nodes. Empirical results show substantial performance gains over sequential execution, with tracing via Extrae/Paraver enabling detailed performance analysis and debugging support. The work provides a practical path to high-performance R analytics in HPC environments and outlines future enhancements including GPU support and expanded data-type handling.

Abstract

R has become a cornerstone of scientific and statistical computing due to its extensive package ecosystem, expressive syntax, and strong support for reproducible analysis. However, as data sizes and computational demands grow, native R parallelism support remains limited. This paper presents RCOMPSs, a scalable runtime system that enables efficient parallel execution of R applications on multicore and manycore systems. RCOMPSs adopts a dynamic, task-based programming model, allowing users to write code in a sequential style, while the runtime automatically handles asynchronous task execution, dependency tracking, and scheduling across available resources. We present RCOMPSs using three representative data analysis algorithms, i.e., K-nearest neighbors (KNN) classification, K-means clustering, and linear regression and evaluate their performance on two modern HPC systems: KAUST Shaheen-III and Barcelona Supercomputing Center (BSC) MareNostrum 5. Experimental results reveal that RCOMPSs demonstrates both strong and weak scalability on up to 128 cores per node and across 32 nodes. For KNN and K-means, parallel efficiency remains above 70% in most settings, while linear regression maintains acceptable performance under shared and distributed memory configurations despite its deeper task dependencies. Overall, RCOMPSs significantly enhances the parallel capabilities of R with minimal, automated, and runtime-aware user intervention, making it a practical solution for large-scale data analytics in high-performance environments.

RCOMPSs: A Scalable Runtime System for R Code Execution on Manycore Systems

TL;DR

RCOMPSs tackles the scalability gap in R by introducing a COMPSs-based runtime binding that lets R developers write sequential code while the runtime exposes tasks, builds a dynamic DAG, and orchestrates distributed execution. The approach yields automatic data movement, fault tolerance, and back-end agnosticism, demonstrated through KNN, K-means, and linear regression on two HPC systems with strong and weak scalability up to 128 cores per node and 32 nodes. Empirical results show substantial performance gains over sequential execution, with tracing via Extrae/Paraver enabling detailed performance analysis and debugging support. The work provides a practical path to high-performance R analytics in HPC environments and outlines future enhancements including GPU support and expanded data-type handling.

Abstract

R has become a cornerstone of scientific and statistical computing due to its extensive package ecosystem, expressive syntax, and strong support for reproducible analysis. However, as data sizes and computational demands grow, native R parallelism support remains limited. This paper presents RCOMPSs, a scalable runtime system that enables efficient parallel execution of R applications on multicore and manycore systems. RCOMPSs adopts a dynamic, task-based programming model, allowing users to write code in a sequential style, while the runtime automatically handles asynchronous task execution, dependency tracking, and scheduling across available resources. We present RCOMPSs using three representative data analysis algorithms, i.e., K-nearest neighbors (KNN) classification, K-means clustering, and linear regression and evaluate their performance on two modern HPC systems: KAUST Shaheen-III and Barcelona Supercomputing Center (BSC) MareNostrum 5. Experimental results reveal that RCOMPSs demonstrates both strong and weak scalability on up to 128 cores per node and across 32 nodes. For KNN and K-means, parallel efficiency remains above 70% in most settings, while linear regression maintains acceptable performance under shared and distributed memory configurations despite its deeper task dependencies. Overall, RCOMPSs significantly enhances the parallel capabilities of R with minimal, automated, and runtime-aware user intervention, making it a practical solution for large-scale data analytics in high-performance environments.
Paper Structure (22 sections, 7 equations, 10 figures, 1 table)

This paper contains 22 sections, 7 equations, 10 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. RCOMPSs Programming Model
  4. COMPSs
  5. RCOMPSs Architecture
  6. RCOMPSs API and Packaging
  7. C++ Integration via Rcpp Package:
  8. R Workers Deployment:
  9. Serialization/Deserialization:
  10. Tracing and Profiling RCOMPSs Execution:
  11. Application Debug Support:
  12. RCOMPSs in Practice
  13. Benchmarking Applications
  14. K-Nearest Neighbors (KNN) Classification
  15. K-means Clustering
  16. ...and 7 more sections

Figures (10)

  • Figure 1: Overview of the COMPSs and RCOMPSs runtime. (a) COMPSs enabling Java, C/C++, Python, and R applications via language-specific plugins.
  • Figure 2: RCOMPSs code of adding four numbers using an add() function, adds just two numbers at a time, and the resulting DAG.
  • Figure 3: DAG of the parallel KNN algorithm.
  • Figure 4: DAG of the parallel K-means algorithm (one iteration).
  • Figure 5: DAG of the parallel linear regression algorithm.
  • ...and 5 more figures