RBoard: A Unified Platform for Reproducible and Reusable Recommender System Benchmarks
Xinyang Shao, Edoardo D'Amico, Gabor Fodor, Tri Kurniawan Wijaya
TL;DR
This paper addresses the reproducibility and comparability gap in recommender system benchmarking. It proposes RBoard, a unified, task-agnostic platform that standardizes data handling, evaluation, and code reuse across CTR prediction, Top-N, and other tasks. The framework enables multi-dataset evaluation, open-source submission code, and a public leaderboard to foster fair comparisons and rapid progress. By bridging academia and industry with reusable pipelines and transparent experiments, RBoard aims to establish a new standard for recommender system benchmarking.
Abstract
Recommender systems research lacks standardized benchmarks for reproducibility and algorithm comparisons. We introduce RBoard, a novel framework addressing these challenges by providing a comprehensive platform for benchmarking diverse recommendation tasks, including CTR prediction, Top-N recommendation, and others. RBoard's primary objective is to enable fully reproducible and reusable experiments across these scenarios. The framework evaluates algorithms across multiple datasets within each task, aggregating results for a holistic performance assessment. It implements standardized evaluation protocols, ensuring consistency and comparability. To facilitate reproducibility, all user-provided code can be easily downloaded and executed, allowing researchers to reliably replicate studies and build upon previous work. By offering a unified platform for rigorous, reproducible evaluation across various recommendation scenarios, RBoard aims to accelerate progress in the field and establish a new standard for recommender systems benchmarking in both academia and industry. The platform is available at https://rboard.org and the demo video can be found at https://bit.ly/rboard-demo.