Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Linked Array Tree: A Constant-Time Search Structure for Big Data

Songpeng Liu

TL;DR

The paper addresses the challenge of fast search in big data by proposing the Linked Array Tree (LAT), a radix-based multi-level structure designed for constant-time access with low memory overhead. LAT stores values on a data level and uses index-level pointers, omitting explicit key storage, with search paths determined by remainders and bitwise operations when the radix is a power of two. The authors provide time-complexity analyses showing constant-time-like behavior for fixed radix and height, and discuss optimizations that leverage bitwise calculations. Empirical comparisons against red-black trees and B+-trees indicate strong performance for data-intensive workloads and lower memory usage, while highlighting trade-offs for sparse data and suggesting parallelization potential and variants (e.g., asymmetric radixes) to balance time and space. The work includes implementation details and a public repository, positioning LAT as a scalable option for large-scale memory and disk management in dense data regimes.

Abstract

As data volumes continue to grow rapidly, traditional search algorithms, like the red-black tree and B+ Tree, face increasing challenges in performance, especially in big data scenarios with intensive storage access. This paper presents the Linked Array Tree (LAT), a novel data structure designed to achieve constant-time complexity for search, insertion, and deletion operations. LAT leverages a sparse, non-moving hierarchical layout that enables direct access paths without requiring rebalancing or data movement. Its low memory overhead and avoidance of pointer-heavy structures make it well-suited for large-scale and intensive workloads. While not specifically tested under parallel or concurrent conditions, the structure's static layout and non-interfering operations suggest potential advantages in such environments. This paper first introduces the structure and algorithms of LAT, followed by a detailed analysis of its time complexity in search, insertion, and deletion operations. Finally, it presents experimental results across both data-intensive and sparse usage scenarios to evaluate LAT's practical performance.

Linked Array Tree: A Constant-Time Search Structure for Big Data

TL;DR

The paper addresses the challenge of fast search in big data by proposing the Linked Array Tree (LAT), a radix-based multi-level structure designed for constant-time access with low memory overhead. LAT stores values on a data level and uses index-level pointers, omitting explicit key storage, with search paths determined by remainders and bitwise operations when the radix is a power of two. The authors provide time-complexity analyses showing constant-time-like behavior for fixed radix and height, and discuss optimizations that leverage bitwise calculations. Empirical comparisons against red-black trees and B+-trees indicate strong performance for data-intensive workloads and lower memory usage, while highlighting trade-offs for sparse data and suggesting parallelization potential and variants (e.g., asymmetric radixes) to balance time and space. The work includes implementation details and a public repository, positioning LAT as a scalable option for large-scale memory and disk management in dense data regimes.

Abstract

As data volumes continue to grow rapidly, traditional search algorithms, like the red-black tree and B+ Tree, face increasing challenges in performance, especially in big data scenarios with intensive storage access. This paper presents the Linked Array Tree (LAT), a novel data structure designed to achieve constant-time complexity for search, insertion, and deletion operations. LAT leverages a sparse, non-moving hierarchical layout that enables direct access paths without requiring rebalancing or data movement. Its low memory overhead and avoidance of pointer-heavy structures make it well-suited for large-scale and intensive workloads. While not specifically tested under parallel or concurrent conditions, the structure's static layout and non-interfering operations suggest potential advantages in such environments. This paper first introduces the structure and algorithms of LAT, followed by a detailed analysis of its time complexity in search, insertion, and deletion operations. Finally, it presents experimental results across both data-intensive and sparse usage scenarios to evaluate LAT's practical performance.

Paper Structure

This paper contains 15 sections, 5 theorems, 10 equations, 13 figures.

Table of Contents

  1. Introduction
  2. Problems when Searching in Big Data
  3. Related Work
  4. Linked Array Tree
  5. Search
  6. Insertion
  7. Deletion
  8. Optimization
  9. Key Properties of LAT
  10. Time Complexity Analysis
  11. Search
  12. Insertion
  13. Deletion
  14. Performance Comparison
  15. Conclusion

Key Result

Lemma 2.1

Let $r$ be the $radix$, $h$ be the $height$, $m$ be the max size of the LAT, then $m=r^h$.

Figures (13)

  • Figure 1: A Linked Array Tree whose $radix$ is 4 and $height$ is 4.
  • Figure 2: Changing the max size of an LAT.
  • Figure 3: Finding the left data array.
  • Figure 4: Optimized remainders calculation.
  • Figure 5: Worst case and best case in finding left data array.
  • ...and 8 more figures

Theorems & Definitions (10)

  • Lemma 2.1
  • proof
  • Lemma 2.2
  • proof
  • Theorem 3.1
  • proof
  • Theorem 3.2
  • proof
  • Theorem 3.3
  • proof