Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

I/O complexity and pebble games with partial computations

Aleksandros Sobczyk

TL;DR

This work studies a generalization of the original Red-Blue Pebble Game to allow arbitrary in-degrees, that can be larger than the size of the fast memory, to minimize the I/O operations by allowing the computation of partial results in the fast memory.

Abstract

Optimizing data movements during program executions is essential for achieving high performance in modern computing systems. This has been classically modeled with the Red-Blue Pebble Game and its variants. In the existing models, it is typically assumed that the number of red pebbles, i.e., the size of the fast memory, is larger than the maximum in-degree in the computational graph (e.g. an arithmetic circuit). This assumption can be restrictive for many real applications, especially when dealing with "big data" in Machine Learning and Scientific Computing. In this work we study a generalization of the original Red-Blue Pebble Game to allow arbitrary in-degrees, that can be larger than the size of the fast memory. The objective is to minimize the I/O operations by allowing the computation of partial results in the fast memory. We show that this variant of the problem is NP-complete, even for the special case where the computational graph consists of a single level, and only two words fit in the fast memory. Approximation algorithms for a couple of special cases are also outlined.

I/O complexity and pebble games with partial computations

TL;DR

This work studies a generalization of the original Red-Blue Pebble Game to allow arbitrary in-degrees, that can be larger than the size of the fast memory, to minimize the I/O operations by allowing the computation of partial results in the fast memory.

Abstract

Optimizing data movements during program executions is essential for achieving high performance in modern computing systems. This has been classically modeled with the Red-Blue Pebble Game and its variants. In the existing models, it is typically assumed that the number of red pebbles, i.e., the size of the fast memory, is larger than the maximum in-degree in the computational graph (e.g. an arithmetic circuit). This assumption can be restrictive for many real applications, especially when dealing with "big data" in Machine Learning and Scientific Computing. In this work we study a generalization of the original Red-Blue Pebble Game to allow arbitrary in-degrees, that can be larger than the size of the fast memory. The objective is to minimize the I/O operations by allowing the computation of partial results in the fast memory. We show that this variant of the problem is NP-complete, even for the special case where the computational graph consists of a single level, and only two words fit in the fast memory. Approximation algorithms for a couple of special cases are also outlined.

Paper Structure

This paper contains 10 sections, 3 theorems, 10 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivation and contributions
  3. Outline
  4. Basic definitions and model of computation
  5. Pebble game and hardness
  6. Hardness
  7. Approximation algorithms for some special cases
  8. An approximation algorithm based on Christofides
  9. A more powerful model
  10. Conclusion

Key Result

Proposition 2.1

Under Assumption def:static_cache_line, a DAG $G=(V,E)$ in the $(M,B)$-I/O model can be uniquely transformed to an equivalent DAG $G'=(V',E')$ in the $(M,1)$-I/O model in polynomial time.

Figures (5)

  • Figure 1: (a) DAG for the summation example with max in-degree three. (b)-(d) three equivalent transformations with max in-degree two.
  • Figure 2: The steps required to create the directed bipartite graph $G'$ from the graph $G$ in the proof of Theorem \ref{['theorem:mod_rbpg_p_hardness']}.
  • Figure 3: Three different possible configurations of two edges in the bipartite graph, and the associated cost in pebble moves to eliminate the right-most edge, assuming the left-most edge was just eliminated.
  • Figure 4: A bipartite graph $G$ and the corresponding graph $G'$ to solve the minimum Hamiltonian Path.
  • Figure 5: The bipartite graph $G$ of Figure \ref{['fig:mod_rbpg_reduction_to_min_hamiltonian_path']} and the corresponding graph $G'$ to solve the minimum Hamiltonian Path in the extended model. Unlike Figure \ref{['fig:mod_rbpg_reduction_to_min_hamiltonian_path']}, the graph $G'$ has edge weights equal to $1$ and $2$, instead of $\{1,2,3\}$.

Theorems & Definitions (9)

  • Proposition 2.1
  • proof
  • Definition 2.1: Machine instructions
  • Definition 3.1: Red-Blue Pebble Game with partial computations
  • Theorem 3.1
  • proof
  • proof
  • Proposition 4.1
  • proof