LatticeHashForest: An Efficient Data Structure for Repetitive Data and Operations
Anamitra Ghorui, Uday P. Khedker
TL;DR
LatticeHashForest (LHF) addresses the combinatorial blow-up in whole-program analyses, notably pointer analysis, by introducing a deduplicating, memoized data structure that assigns unique identifiers to aggregate data and memoizes operations on these identifiers. LHF supports nested constructions, enables incremental lattice-like growth, and leverages a virtual-memory–inspired abstraction to efficiently represent and reuse data across contexts. The approach is demonstrated within the LFCPA/VASCO framework and shows substantial memory reductions and speedups for large-scale inputs, while providing an abstract model and practical C++ implementation. The work also discusses comparisons with related structures (BDDs, ZDDs, union-find, e-graphs) and outlines potential extensions, parallelization, and disk-backed memory strategies. Overall, LHF offers a generic, efficient mechanism to exploit data redundancy in complex program analyses, with broad applicability beyond pointer analysis.
Abstract
Analysis of entire programs as a single unit, or whole-program analysis, involves propagation of large amounts of information through the control flow of the program. This is especially true for pointer analysis, where, unless significant compromises are made in the precision of the analysis, there is a combinatorial blowup of information. One of the key problems we observed in our own efforts to this end is that a lot of duplicate data was being propagated, and many low-level data structure operations were repeated a large number of times. We present what we consider to be a novel and generic data structure, LatticeHashForest (LHF), to store and operate on such data in a manner that eliminates a majority of redundant computations and duplicate data in scenarios similar to those encountered in compilers and program optimization. LHF differs from similar work in this vein, such as hash-consing, ZDDs, and BDDs, by not only providing a way to efficiently operate on large, aggregate structures, but also modifying the elements of such structures in a manner that they can be deduplicated immediately. LHF also provides a way to perform a nested construction of elements such that they can be deduplicated at multiple levels, cutting down the need for additional, nested computations. We provide a detailed structural description, along with an abstract model of this data structure. An entire C++ implementation of LHF is provided as an artifact along with evaluations of LHF using examples and benchmark programs. We also supply API documentation and a user manual for users to make independent applications of LHF. Our main use case in the realm of pointer analysis shows memory usage reduction to an almost negligible fraction, and speedups beyond 4x for input sizes approaching 10 million when compared to other implementations.