Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Theoretically Grounded Pruning of Large Ground Sets for Constrained, Discrete Optimization

Ankur Nath, Alan Kuhnle

TL;DR

This work develops light-weight pruning algorithms to quickly discard elements that are unlikely to be part of an optimal solution and outperforms state-of-the-art classical and machine learning heuristics for pruning.

Abstract

Modern instances of combinatorial optimization problems often exhibit billion-scale ground sets, which have many uninformative or redundant elements. In this work, we develop light-weight pruning algorithms to quickly discard elements that are unlikely to be part of an optimal solution. Under mild assumptions on the instance, we prove theoretical guarantees on the fraction of the optimal value retained and the size of the resulting pruned ground set. Through extensive experiments on real-world datasets for various applications, we demonstrate that our algorithm, QuickPrune, efficiently prunes over 90% of the ground set and outperforms state-of-the-art classical and machine learning heuristics for pruning.

Theoretically Grounded Pruning of Large Ground Sets for Constrained, Discrete Optimization

TL;DR

This work develops light-weight pruning algorithms to quickly discard elements that are unlikely to be part of an optimal solution and outperforms state-of-the-art classical and machine learning heuristics for pruning.

Abstract

Modern instances of combinatorial optimization problems often exhibit billion-scale ground sets, which have many uninformative or redundant elements. In this work, we develop light-weight pruning algorithms to quickly discard elements that are unlikely to be part of an optimal solution. Under mild assumptions on the instance, we prove theoretical guarantees on the fraction of the optimal value retained and the size of the resulting pruned ground set. Through extensive experiments on real-world datasets for various applications, we demonstrate that our algorithm, QuickPrune, efficiently prunes over 90% of the ground set and outperforms state-of-the-art classical and machine learning heuristics for pruning.

Paper Structure

This paper contains 30 sections, 8 theorems, 7 equations, 8 figures, 4 tables, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Preliminaries and Notation
  4. QUICKPRUNE: A PRUNING ALGORITHM WITH GUARANTEES
  5. Pruning for a Single Knapsack Constraint
  6. Proof of Theorem \ref{['thm:single']}
  7. The Pruning Algorithm: QuickPrune
  8. EMPIRICAL EVALUATION
  9. Set 1: Evaluation on Size Constraints
  10. Classical Problems on Graph.
  11. Image Retrieval System Results.
  12. Set 2: Performance on Knapsack Constraint
  13. Video Retrieval System Results.
  14. Set 3: Multi-Budget and Single-Budget Comparison
  15. CONCLUDING REMARKS AND FUTURE DIRECTIONS
  16. ...and 15 more sections

Key Result

Theorem 1

Let $\kappa_{\min} < \kappa_{\max}$, let $0 < \eta \le 1/2$, and let $f, c$ be $\gamma$-submodular and modular functions, respectively. Suppose that for all $\kappa' \in [\kappa_{\min}, \kappa_{\max}]$, the instance $(f,c,\kappa')$ satisfies the NHI assumption with $\eta$. Let $\mathcal{U}'$ be the

Figures (8)

  • Figure 1: (a): Typical empirical results of QuickPrune versus competing methods on an instance of the MaximumCover problem: QuickPrune retains 99.99% of the optimal value while pruning $99.95\%$ of the ground set. (b): Plot of the pruning ratio $\alpha (\varepsilon, \delta, \gamma )$ of Theorem \ref{['thm:multi']}, as a function of the parameter $\delta$ of QuickPrune and the $\gamma$-submodularity of the objective function $f$. Here, $\varepsilon$ is fixed to $0.01$.
  • Figure 2: Comparison of the number of oracle calls between SS and QuickPrune.
  • Figure 3: Comparison between QuickPrune and QuickPrune-Single on a selection of data sets for IM.
  • Figure 4: Average GPU and CPU memory utilization among algorithms for MaxCover on YouTube dataset.
  • Figure 5: Retrieval system: (a) The pipeline of the retrieval system and a visual representation of images selected by various algorithms for the Beans Dataset. (b-e) Multi-budget analysis of the retrieval system for budgets ranging from $5$ to $20$. Note that $P_g$ represents the percentage of the ground set that has been pruned.
  • ...and 3 more figures

Theorems & Definitions (16)

  • Theorem 1
  • Theorem 2
  • Proposition 1
  • Proposition 2
  • Lemma 1
  • Proposition 3
  • Proposition 4
  • proof : Proof of Theorem \ref{['thm:single']}
  • Proposition 5
  • proof : Proof of Theorem \ref{['thm:multi']}
  • ...and 6 more