Accelerating ERM for data-driven algorithm design using output-sensitive techniques

TL;DR

This work begins the study of techniques to develop efficient ERM learning algorithms for data-driven algorithm design by enumerating the pieces of the sum dual loss functions for a collection of problem instances.

Abstract

Data-driven algorithm design is a promising, learning-based approach for beyond worst-case analysis of algorithms with tunable parameters. An important open problem is the design of computationally efficient data-driven algorithms for combinatorial algorithm families with multiple parameters. As one fixes the problem instance and varies the parameters, the "dual" loss function typically has a piecewise-decomposable structure, i.e. is well-behaved except at certain sharp transition boundaries. In this work we initiate the study of techniques to develop efficient ERM learning algorithms for data-driven algorithm design by enumerating the pieces of the sum dual loss functions for a collection of problem instances. The running time of our approach scales with the actual number of pieces that appear as opposed to worst case upper bounds on the number of pieces. Our approach involves two novel ingredients -- an output-sensitive algorithm for enumerating polytopes induced by a set of hyperplanes using tools from computational geometry, and an execution graph which compactly represents all the states the algorithm could attain for all possible parameter values. We illustrate our techniques by giving algorithms for pricing problems, linkage-based clustering and dynamic-programming based sequence alignment.

Figures (9)

• Figure 1: The first three levels of an example execution tree of a clustering instance on four points, with a two-parameter algorithm ($\mathcal{P} = \blacktriangle^2$). Successive partitions $\mathcal{P}_0$, $\mathcal{P}_1$, $\mathcal{P}_2$ are shown at merge levels $0$, $1$, and $2$, respectively, and the nested shapes show cluster merges.
• Figure 2: Incoming nodes used for computing the pieces at the node $P[i][j]$ in the execution DAG.
• Figure 3: ComputeCompactExecutionDAG
• Figure 4: AugmentedClarkson($z,H=(A,b),c$)
• Figure 5: RayShoot

Theorems & Definitions (73)

• Definition 1: Piecewise structured with linear boundaries
• Definition 2: Output-polynomial Fixed Parameter Tractable
• Theorem 3.1: Clarkson's algorithm
• Definition 3: Cell adjacency graph
• Theorem 3.2
• proof
• Corollary 3.3
• proof
• Example 1
• Example 2