Hammer: Towards Efficient Hot-Cold Data Identification via Online Learning

TL;DR

This work proposes a novel solution based on online learning strategies that dynamically adapts to changing data access patterns, achieving higher accuracy and lower operational costs.

Abstract

Efficient management of storage resources in big data and cloud computing environments requires accurate identification of data's "cold" and "hot" states. Traditional methods, such as rule-based algorithms and early AI techniques, often struggle with dynamic workloads, leading to low accuracy, poor adaptability, and high operational overhead. To address these issues, we propose a novel solution based on online learning strategies. Our approach dynamically adapts to changing data access patterns, achieving higher accuracy and lower operational costs. Rigorous testing with both synthetic and real-world datasets demonstrates a significant improvement, achieving a 90% accuracy rate in hot-cold classification. Additionally, the computational and storage overheads are considerably reduced.

Figures (3)

• Figure 1: Hammer system overview.
• Figure 2: Online prediction process.
• Figure 3: Online evaluation strategy based on Sketch-min counting.