Finding Optimal Video Moment without Training: Gaussian Boundary Optimization for Weakly Supervised Video Grounding

TL;DR

The paper tackles weakly supervised video grounding by reframing boundary prediction as a principled inference problem. It introduces Gaussian Boundary Optimization (GBO), which maximizes proposal coverage while penalizing segment length, and derives a closed-form solution across regimes of the penalty weight $\lambda$. GBO is training-free and compatible with both single-Gaussian and Gaussian mixture proposals, delivering consistent improvements over baselines on ActivityNet Captions and Charades-STA with negligible overhead. The work offers clear interpretability of boundary behavior through explicit formulas and demonstrates broad generalization beyond Gaussian proposals, establishing a practical, model-agnostic enhancement for temporal grounding.

Abstract

Weakly supervised temporal video grounding aims to localize query-relevant segments in untrimmed videos using only video-sentence pairs, without requiring ground-truth segment annotations that specify exact temporal boundaries. Recent approaches tackle this task by utilizing Gaussian-based temporal proposals to represent query-relevant segments. However, their inference strategies rely on heuristic mappings from Gaussian parameters to segment boundaries, resulting in suboptimal localization performance. To address this issue, we propose Gaussian Boundary Optimization (GBO), a novel inference framework that predicts segment boundaries by solving a principled optimization problem that balances proposal coverage and segment compactness. We derive a closed-form solution for this problem and rigorously analyze the optimality conditions under varying penalty regimes. Beyond its theoretical foundations, GBO offers several practical advantages: it is training-free and compatible with both single-Gaussian and mixture-based proposal architectures. Our experiments show that GBO significantly improves localization, achieving state-of-the-art results across standard benchmarks. Extensive experiments demonstrate the efficiency and generalizability of GBO across various proposal schemes. The code is available at \href{https://github.com/sunoh-kim/gbo}{https://github.com/sunoh-kim/gbo}.

Figures (6)

• Figure 1: Weakly supervised video grounding. Compared to previous methods based on heuristic segment prediction, our Gaussian boundary optimization formulates the segment prediction as a principled optimization problem. By maximizing coverage under a Gaussian-based proposal while penalizing excessive segment length via a penalty weight, our method yields segments that more accurately align with the query-relevant content.
• Figure 2: Performance improvement by applying our Gaussian Boundary Optimization to existing Gaussian proposal-based methods. Our method yields significant gains over both CPL and PPS baselines, improving performance by up to +8.09%p on ActivityNet Captions and +6.52%p on Charades-STA, respectively.
• Figure 3: Overview of the proposed Gaussian Boundary Optimization (GBO) framework for video grounding. We first generate Gaussian-based temporal proposals characterized by center and width. Unlike prior heuristic strategies that map proposals to segments with limited proposal information, GBO formulates boundary prediction as an optimization problem balancing proposal coverage and segment compactness. This principled approach fully leverages proposal information, yielding improved accuracy while remaining fast, training-free, and model-agnostic.
• Figure 4: Performance comparison of segment predictions under different penalty weights $\lambda$. The left and right columns correspond to ActivityNet Captions and Charades-STA, respectively. In each column, the top row shows results from GBOCNM, while the two bottom rows show results from GBOCPL at Rank@1 and Rank@5, respectively. Green, purple, and orange dots indicate performance at IoU=0.5, IoU=0.7, and mean IoU, respectively.
• Figure 5: Performance comparison of segment predictions under different penalty weights $\lambda$. The left and right columns correspond to ActivityNet Captions and Charades-STA, respectively. The two rows show results from GBOPPS at Rank@1 and Rank@5, respectively. Green, purple, and orange dots indicate performance at IoU=0.5, IoU=0.7, and mean IoU, respectively.

Theorems & Definitions (1)

• Theorem 1: Optimal Segment Prediction