Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Weakly Supervised Video Grounding via Diverse Inference Strategies for Boundary and Prediction Selection

Sunoh Kim, Daeho Um

TL;DR

This paper tackles weakly supervised video grounding by highlighting the underexplored role of inference strategies. It extends Gaussian-based proposals with diverse boundary prediction methods and loss-aware top-1 selection to better localize query-relevant segments ($s$, $e$) without additional training. Five boundary strategies and four top-1 strategies are evaluated, with Shortest Tail and IoU+LossMax (or IoU+LossSum per dataset) emerging as top configurations. Across ActivityNet Captions and Charades-STA, the proposed inference strategies yield clear gains over baselines, demonstrating practical, training-free improvements for boundary localization and proposal selection in complex videos.

Abstract

Weakly supervised video grounding aims to localize temporal boundaries relevant to a given query without explicit ground-truth temporal boundaries. While existing methods primarily use Gaussian-based proposals, they overlook the importance of (1) boundary prediction and (2) top-1 prediction selection during inference. In their boundary prediction, boundaries are simply set at half a standard deviation away from a Gaussian mean on both sides, which may not accurately capture the optimal boundaries. In the top-1 prediction process, these existing methods rely heavily on intersections with other proposals, without considering the varying quality of each proposal. To address these issues, we explore various inference strategies by introducing (1) novel boundary prediction methods to capture diverse boundaries from multiple Gaussians and (2) new selection methods that take proposal quality into account. Extensive experiments on the ActivityNet Captions and Charades-STA datasets validate the effectiveness of our inference strategies, demonstrating performance improvements without requiring additional training.

Enhancing Weakly Supervised Video Grounding via Diverse Inference Strategies for Boundary and Prediction Selection

TL;DR

This paper tackles weakly supervised video grounding by highlighting the underexplored role of inference strategies. It extends Gaussian-based proposals with diverse boundary prediction methods and loss-aware top-1 selection to better localize query-relevant segments (, ) without additional training. Five boundary strategies and four top-1 strategies are evaluated, with Shortest Tail and IoU+LossMax (or IoU+LossSum per dataset) emerging as top configurations. Across ActivityNet Captions and Charades-STA, the proposed inference strategies yield clear gains over baselines, demonstrating practical, training-free improvements for boundary localization and proposal selection in complex videos.

Abstract

Weakly supervised video grounding aims to localize temporal boundaries relevant to a given query without explicit ground-truth temporal boundaries. While existing methods primarily use Gaussian-based proposals, they overlook the importance of (1) boundary prediction and (2) top-1 prediction selection during inference. In their boundary prediction, boundaries are simply set at half a standard deviation away from a Gaussian mean on both sides, which may not accurately capture the optimal boundaries. In the top-1 prediction process, these existing methods rely heavily on intersections with other proposals, without considering the varying quality of each proposal. To address these issues, we explore various inference strategies by introducing (1) novel boundary prediction methods to capture diverse boundaries from multiple Gaussians and (2) new selection methods that take proposal quality into account. Extensive experiments on the ActivityNet Captions and Charades-STA datasets validate the effectiveness of our inference strategies, demonstrating performance improvements without requiring additional training.

Paper Structure

This paper contains 29 sections, 9 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Sliding window-based methods
  4. Reconstruction-based methods
  5. Proposed Method
  6. Boundary Prediction
  7. Long Tail
  8. Short Tail
  9. Shortest Tail
  10. Average
  11. Attention
  12. Top-1 Prediction Selection
  13. IoU
  14. Loss
  15. IoU+LossSum
  16. ...and 14 more sections

Figures (2)

  • Figure 1: Overview of the proposed inference process, consisting of boundary prediction strategies and top-1 prediction selection strategies. To improve the performance of Gaussian proposal-based methods, we explore five strategies for boundary prediction and four strategies for top-1 prediction selection to derive an optimal combination of inference strategies.
  • Figure 2: Performance improvements of our method over the baseline.