Latency Based Tiling

TL;DR

The paper addresses the challenge of efficiently tiling affine programs to exploit memory hierarchies without incurring prohibitive autotuning costs. It introduces Latency Based Tiling, a method that derives tile sizes from empirical latency measurements using triangular loop profiling to capture miss ratio scaling and infer cache boundaries, including $L1$, $L2$, and $L3$ caches, via the metric $MRS$. The approach yields a hardware-agnostic, Rust-implemented tiling strategy that applies to a subset of the polyhedral model and emphasizes defensive tiling in shared environments, achieving up to $1.21\times$ speedups on PolyBench benchmarks with negligible compile-time overhead. This work provides a portable, safe, and fast alternative to auto-tuning for affine-program tiling, with practical impact for multi-core, multi-process systems where cache sharing influences performance; future directions include automatic mid-end passes and unrolling/vectorization to further boost locality.

Abstract

Latency Based Tiling provides a systems based approach to deriving approximate tiling solution that maximizes locality while maintaining a fast compile time. The method uses triangular loops to characterize miss ratio scaling of a machine avoiding prefetcher distortion. Miss ratio scaling captures the relationship between data access latency and working set size with sharp increases in latency indicating the data footprint exceeds capacity from a cache level. Through these noticeable increases in latency we can determine an approximate location for L1, L2, and L3 memory sizes. These sizes are expected to be under approximations of a systems true memory sizes which is in line with our expectations given the shared nature of cache in a multi process system as described in defensive loop tiling. Unlike auto tuning, which can be effective but prohibitively slow, Latency Based Tiling achieves negligible compile time overhead. The implementation in Rust enables a hardware agnostic approach which combined with a cache timing based techniques, yields a portable, memory safe system running wherever Rust is supported. The tiling strategy is applied to a subset of the polyhedral model, where loop nestings are tiled based on both the derived memory hierarchy and the observed data footprint per iteration.

Figures (7)

• Figure 1: Key measurement pattern implementation in Rust
• Figure 2: Core data structures for latency analysis
• Figure 3: Section of 3mm without tiling insertion.
• Figure 4: Section of 3mm with tiling insertion.
• Figure 5: Data Traversal Latency with Approximate Cache Levels