Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LakeMLB: Data Lake Machine Learning Benchmark

Feiyu Pan, Tianbin Zhang, Aoqian Zhang, Yu Sun, Zheng Wang, Lixing Chen, Li Pan, Jianhua Li

TL;DR

LakeMLB introduces the first benchmark focused on evaluating multi-table machine learning within data lake environments, emphasizing Union and Join scenarios with three real-world datasets per scenario. It formalizes table unionability and joinability, and provides end-to-end evaluation across three integration strategies: pre-training-based, data augmentation-based, and feature augmentation-based approaches. Comprehensive experiments show that pre-training excels in Union settings, feature augmentation shines in Join settings, and transfer-learning methods offer robust performance across tasks, highlighting the importance of semantic relationships and high-quality row matching. By releasing datasets and code, LakeMLB aims to standardize evaluation and spur the development of data-lake-aware learning techniques for lakehouse ecosystems.

Abstract

Modern data lakes have emerged as foundational platforms for large-scale machine learning, enabling flexible storage of heterogeneous data and structured analytics through table-oriented abstractions. Despite their growing importance, standardized benchmarks for evaluating machine learning performance in data lake environments remain scarce. To address this gap, we present LakeMLB (Data Lake Machine Learning Benchmark), designed for the most common multi-source, multi-table scenarios in data lakes. LakeMLB focuses on two representative multi-table scenarios, Union and Join, and provides three real-world datasets for each scenario, covering government open data, finance, Wikipedia, and online marketplaces. The benchmark supports three representative integration strategies: pre-training-based, data augmentation-based, and feature augmentation-based approaches. We conduct extensive experiments with state-of-the-art tabular learning methods, offering insights into their performance under complex data lake scenarios. We release both datasets and code to facilitate rigorous research on machine learning in data lake ecosystems; the benchmark is available at https://github.com/zhengwang100/LakeMLB.

LakeMLB: Data Lake Machine Learning Benchmark

TL;DR

LakeMLB introduces the first benchmark focused on evaluating multi-table machine learning within data lake environments, emphasizing Union and Join scenarios with three real-world datasets per scenario. It formalizes table unionability and joinability, and provides end-to-end evaluation across three integration strategies: pre-training-based, data augmentation-based, and feature augmentation-based approaches. Comprehensive experiments show that pre-training excels in Union settings, feature augmentation shines in Join settings, and transfer-learning methods offer robust performance across tasks, highlighting the importance of semantic relationships and high-quality row matching. By releasing datasets and code, LakeMLB aims to standardize evaluation and spur the development of data-lake-aware learning techniques for lakehouse ecosystems.

Abstract

Modern data lakes have emerged as foundational platforms for large-scale machine learning, enabling flexible storage of heterogeneous data and structured analytics through table-oriented abstractions. Despite their growing importance, standardized benchmarks for evaluating machine learning performance in data lake environments remain scarce. To address this gap, we present LakeMLB (Data Lake Machine Learning Benchmark), designed for the most common multi-source, multi-table scenarios in data lakes. LakeMLB focuses on two representative multi-table scenarios, Union and Join, and provides three real-world datasets for each scenario, covering government open data, finance, Wikipedia, and online marketplaces. The benchmark supports three representative integration strategies: pre-training-based, data augmentation-based, and feature augmentation-based approaches. We conduct extensive experiments with state-of-the-art tabular learning methods, offering insights into their performance under complex data lake scenarios. We release both datasets and code to facilitate rigorous research on machine learning in data lake ecosystems; the benchmark is available at https://github.com/zhengwang100/LakeMLB.
Paper Structure (42 sections, 4 figures, 4 tables)

This paper contains 42 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and Task Setting
  3. Table Unionability
  4. Table Joinability
  5. Machine Learning in Data Lakes
  6. Benchmark Design
  7. Union Case
  8. Join Case
  9. Strengths and Limitations
  10. Strengths.
  11. Limitations.
  12. Task Challenges and Potential Solutions
  13. Task Challenges
  14. Potential Solutions
  15. Learning with Unionable Tables
  16. ...and 27 more sections

Figures (4)

  • Figure 1: Machine learning pipeline in data lakes.
  • Figure 2: Illustration of unionable and joinable tables.
  • Figure 3: Effect of auxiliary-table sample ratio on transfer performance (union setting). Bars report $\Delta$Score relative to the 0% auxiliary-sample baseline (positive/negative values indicate gains/losses).
  • Figure 4: Impact of match quality on accuracy (join setting).