Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Determinantal Point Process as an alternative to NMS

Samik Some, Mithun Das Gupta, Vinay P. Namboodiri

TL;DR

The paper presents a determinantal point process (DPP) based replacement for the first NMS step after the Region Proposal Network in object detection. By constructing an L-ensemble $\mathbf{L}=\alpha [e^{\mathbf{s}} e^{\mathbf{s}^T}] \odot \mathrm{IoU}$ from RPN scores $\mathbf{s}$ and pairwise overlaps $\mathrm{IoU}$, it maximizes $\log\det \mathbf{L}_Y$ via a greedy, submodular optimization to select a diverse subset of proposals. Experiments on MS-COCO and PASCAL VOC show that the DPP-based method achieves competitive or improved AP metrics compared to GreedyNMS and other baselines, with notable gains in IoU-aware recall and in crowded scenes. The approach preserves the post-processing nature of NMS, requires no additional training, and can be plugged into existing pipelines with modest changes, offering a practical route to more diverse and accurate detections. $\log\det$ objectives and the PSD property of $\mathbf{L}$ underpin the theoretical guarantees and stable behavior of the selection process.

Abstract

We present a determinantal point process (DPP) inspired alternative to non-maximum suppression (NMS) which has become an integral step in all state-of-the-art object detection frameworks. DPPs have been shown to encourage diversity in subset selection problems. We pose NMS as a subset selection problem and posit that directly incorporating DPP like framework can improve the overall performance of the object detection system. We propose an optimization problem which takes the same inputs as NMS, but introduces a novel sub-modularity based diverse subset selection functional. Our results strongly indicate that the modifications proposed in this paper can provide consistent improvements to state-of-the-art object detection pipelines.

Determinantal Point Process as an alternative to NMS

TL;DR

The paper presents a determinantal point process (DPP) based replacement for the first NMS step after the Region Proposal Network in object detection. By constructing an L-ensemble from RPN scores and pairwise overlaps , it maximizes via a greedy, submodular optimization to select a diverse subset of proposals. Experiments on MS-COCO and PASCAL VOC show that the DPP-based method achieves competitive or improved AP metrics compared to GreedyNMS and other baselines, with notable gains in IoU-aware recall and in crowded scenes. The approach preserves the post-processing nature of NMS, requires no additional training, and can be plugged into existing pipelines with modest changes, offering a practical route to more diverse and accurate detections. objectives and the PSD property of underpin the theoretical guarantees and stable behavior of the selection process.

Abstract

We present a determinantal point process (DPP) inspired alternative to non-maximum suppression (NMS) which has become an integral step in all state-of-the-art object detection frameworks. DPPs have been shown to encourage diversity in subset selection problems. We pose NMS as a subset selection problem and posit that directly incorporating DPP like framework can improve the overall performance of the object detection system. We propose an optimization problem which takes the same inputs as NMS, but introduces a novel sub-modularity based diverse subset selection functional. Our results strongly indicate that the modifications proposed in this paper can provide consistent improvements to state-of-the-art object detection pipelines.

Paper Structure

This paper contains 11 sections, 3 theorems, 9 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Determinantal Point Processes
  4. Method
  5. Experiments and Results
  6. MS-COCO
  7. PASCAL VOC
  8. Varying the maximum window and scaling parameters
  9. IoU vs Recall
  10. Qualitative Results
  11. Conclusion and Future Work

Key Result

Lemma 1

$\mathbf{L} = \alpha [e^\mathbf{s} e^{\mathbf{s}^T}] \odot \mathbf{IoU}$ is positive semidefinite.

Figures (3)

  • Figure 1: Comparison of varying the maximum window parameter $k$ in our algorithm.
  • Figure 3: IoU vs Recall plot for gNMS$_{400}$ and DPP$_{400}^5$
  • Figure 4: Qualitative results (best viewed in color). Blue boxes are produced by our method DPP$_{400}^{5}$, green boxes are produced by gNMS$_{400}$. Blue dotted cluster represents results where DPP$_{400}^{5}$ performs better than gNMS$_{400}$. Brown cluster represents similar performance. Green cluster represents cases where gNMS$_{400}$ seems to perform better, although the person detection is still superior for DPP$_{400}^{5}$.

Theorems & Definitions (5)

  • Lemma 1
  • proof
  • Lemma 2
  • Lemma 3
  • proof