Brame: Hierarchical Data Management Framework for Cloud-Edge-Device Collaboration

TL;DR

Brame addresses data management in cloud-edge-device collaboration by introducing a Block-based hierarchical framework with a three-tier storage architecture across Cloud, Edge, and End. It replaces tuple-level storage with $Block$s and integrates offline Block Generation and online Scheduling, guided by a temperature model and a $0$-$1$ knapsack formulation for data placement, supported by a $Max-Min$ index and K-D Tree routing. Experiments on two real datasets demonstrate improved read query hit rates and cross-tier data migration efficiency, while reducing cloud-end communication and preserving data locality. The work offers a scalable foundation for CEDC data management and suggests directions for online tuning and multi-table extensions.

Abstract

In the realm of big data, cloud-edge-device collaboration is prevalent in industrial scenarios. However, a systematic exploration of the theory and methodologies related to data management in this field is lacking. This paper delves into the sub-problem of data storage and scheduling within cloud-edge-device collaborative environments. Following extensive research and analysis of the characteristics and requirements of data management in cloud-edge collaboration, it is evident that existing studies on hierarchical data management primarily focus on the migration of hot and cold data. Additionally, these studies encounter challenges such as elevated operational and maintenance costs, difficulties in locating data within tiered storage, and intricate metadata management attributable to excessively fine-grained management granularity. These challenges impede the fulfillment of the storage needs in cloud-edge-device collaboration. To overcome these challenges, we propose a \underline{B}lock-based hie\underline{R}archical d\underline{A}ta \underline{M}anagement fram\underline{E}work, \textbf{Brame}, which advocates for a workload-aware three-tier storage architecture and suggests a shift from using tuples to employing $Blocks$ as the fundamental unit for data management. \textbf{Brame} owns an offline block generation method designed to facilitate efficient block generation and expeditious query routing. Extensive experiments substantiate the superior performance of \textbf{Brame}.

Figures (11)

• Figure 1: Tuple-level vs Block-level.
• Figure 2: An Overview of Brame.
• Figure 3: The Workflow of Brame's Offline Block Generation Technology.
• Figure 4: Workload-aware vs Data-aware Approaches.
• Figure 5: Experimental Results of Cloud-Edge Data Scheduling, on Power