Efficient Hierarchical Storage Management Framework Empowered by Reinforcement Learning

TL;DR

The paper tackles the dynamic data-management challenge posed by big-data and cloud computing by proposing an open-source hierarchical storage system (HSS) that integrates multiple storage frameworks. It introduces an RL-based data-migration policy modeled as a continuous-time MDP/SMDP with a fuzzy rule-based value function and TD($\lambda$) updates, enabling online, adaptive placement of data across tiers. Through both simulation and a cloud-based deployment, the approach is compared to three rule-based baselines and consistently achieves better efficiency and near-optimal data distribution, while incurring fewer migrations. The work demonstrates the practicality of RL for autonomous storage management and provides a scalable framework with broad potential impact on data-intensive applications and cloud storage operations.

Abstract

With the rapid development of big data and cloud computing, data management has become increasingly challenging. Over the years, a number of frameworks for data management and storage with various characteristics and features have become available. Most of these are highly efficient, but ultimately create data silos. It becomes difficult to move and work coherently with data as new requirements emerge as no single framework can efficiently fulfill the data management needs of diverse applications. A possible solution is to design smart and efficient hierarchical (multi-tier) storage solutions. A hierarchical storage system (HSS) is a meta solution that consists of different storage frameworks organized as a jointly constructed large storage pool. It brings a number of benefits including better utilization of the storage, cost-efficiency, and use of different features provided by the underlying storage frameworks. In order to maximize the gains of hierarchical storage solutions, it is important that they include intelligent and autonomous mechanisms for data management grounded in the features of the different underlying frameworks. These decisions should be made according to the characteristics of the dataset, tier status, and access patterns. These are highly dynamic parameters and defining a policy based on the mentioned parameters is a non-trivial task. This paper presents an open-source hierarchical storage framework with a dynamic migration policy based on reinforcement learning (RL). We present a mathematical model, a software architecture, and an implementation based on both simulations and a live cloud-based environment. We compare the proposed RL-based strategy to a baseline of three rule-based policies, showing that the RL-based policy achieves significantly higher efficiency and optimal data distribution in different scenarios compared to the dynamic rule-based policies.

Figures (13)

• Figure 1: Three tier hierarchical storage system.
• Figure 2: Membership function representing the categories 'Large' and 'Small', the value of the function stands for how likely a input is 'Large' or 'Small'.
• Figure 3: Workflow of RL-based policy, formed on an example of three tiers HSS. Initial distribution of the files is not optimal (left side bar plot). After the RL-based policy controller takes actions based on the access pattern in step1 and the self-updating RL agents in step2 and step3, the framework optimally place files in different tiers.
• Figure 4: Structure of the simulation system. Three tier hierarchical storage solution with varying access speed and storage capacities.
• Figure 5: Structure of the cloud distributed system. The architecture is based on three storage tiers, one controller node and multiple front-end nodes to handle requests coming from the clients.