Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Safer Heuristics With XPlain

Pantea Karimi, Solal Pirelli, Siva Kesava Reddy Kakarla, Ryan Beckett, Santiago Segarra, Beibin Li, Pooria Namyar, Behnaz Arzani

TL;DR

XPlain is proposed, a tool that extends heuristic analyzers and helps operators understand when and why their heuristics underperform and presents promising initial results that show such an extension is viable.

Abstract

Many problems that cloud operators solve are computationally expensive, and operators often use heuristic algorithms (that are faster and scale better than optimal) to solve them more efficiently. Heuristic analyzers enable operators to find when and by how much their heuristics underperform. However, these tools do not provide enough detail for operators to mitigate the heuristic's impact in practice: they only discover a single input instance that causes the heuristic to underperform (and not the full set), and they do not explain why. We propose XPlain, a tool that extends these analyzers and helps operators understand when and why their heuristics underperform. We present promising initial results that show such an extension is viable.

Towards Safer Heuristics With XPlain

TL;DR

XPlain is proposed, a tool that extends heuristic analyzers and helps operators understand when and why their heuristics underperform and presents promising initial results that show such an extension is viable.

Abstract

Many problems that cloud operators solve are computationally expensive, and operators often use heuristic algorithms (that are faster and scale better than optimal) to solve them more efficiently. Heuristic analyzers enable operators to find when and by how much their heuristics underperform. However, these tools do not provide enough detail for operators to mitigate the heuristic's impact in practice: they only discover a single input instance that causes the heuristic to underperform (and not the full set), and they do not explain why. We propose XPlain, a tool that extends these analyzers and helps operators understand when and why their heuristics underperform. We present promising initial results that show such an extension is viable.

Paper Structure

This paper contains 14 sections, 1 theorem, 8 equations, 7 figures.

Table of Contents

  1. Introduction
  2. What is heuristic analysis?
  3. The case for comprehensive analysis
  4. Challenges
  5. The $\mathcal{X}$Plain proposal
  6. The domain-specific language
  7. The adversarial subspace generator
  8. The explainer
  9. The generalizer and instance generator
  10. Related work
  11. Acknowledgements
  12. Formalizing $\mathcal{X}$Plain's DSL
  13. $\mathcal{X}$Plain's node description
  14. $\mathcal{X}$Plain can model any linear optimization

Key Result

theorem 1

We can model any linear optimization (linear programming or mixed integer linear programming) as a flow network using the six node behaviors ($\mathcal{N}_{split}$, $\mathcal{N}_{pick}$, $\mathcal{N}_{mult}$, $\mathcal{N}_{allEq}$, and $\mathcal{N}_{sink}$)

Figures (7)

  • Figure 1: Example heuristics and their encoding in MetaOpt (sub-figures (b) and (c)). Heuristic in sub-figure (b) forces the demands less than a threshold to be pinned and then solves a flow maximization problem, heuristic in sub-figure (c) assigns the first bin that can fit the ball.
  • Figure 2: Example adversarial instance for FF with equal-sized bins with size of 1; the optimal uses 8 bins and the heuristic 9.
  • Figure 3: $\mathcal{X}$Plain: the system architecture we propose to extend existing heuristic analyzers.
  • Figure 4: Encoding heuristics in our DSL. We show sink nodes in ; source nodes enforcing behavior of split nodes in and source nodes enforcing behavior of pick nodes in ; copy nodes in ; and split nodes with limited outgoing capacity in . The edge colors show type 2 explanations: more intense red (blue) edges show there are more samples in the subspace that only the heuristic (optimal) uses. In (a), DP uses the shortest path for the demand between $1$$\rightsquigarrow$$3$ and the optimal does not. In (b), we see FF places a large ball ($B_0$) in the first bin, causing it to have to place the last ball differently, too. We used 3000 samples for each explanation. $\mathcal{X}$Plain took 20 minutes to produce each figure.
  • Figure 5: The adversarial subspace generator: (a) finds a rough subspace and separates bad samples () from good ones (); (b) it trains a regression tree on these samples and uses it to refine the subspace and produces (c). We show the first subspace ($D_0$) for our FF example in (c). Here, $C^i_j$ encodes the rough subspace and $T_i$ and $V_i$ the path in the regression tree.
  • ...and 2 more figures

Theorems & Definitions (1)

  • theorem 1