MONA: Moving Object Detection from Videos Shot by Dynamic Camera

TL;DR

We address the challenge of separating camera-induced motion from object motion in dynamic-camera videos. MONA combines Dynamic Points Extraction and Moving Object Segmentation to identify dynamic regions using a probabilistic model $p(\mathbf{x}|V,\mathbf{x}_q)$, adaptive frame-wise thresholding with $\bar{m}_t$, and an area-proportional bounding-box filter to prompt SAM for precise segmentation. Evaluation via integration with LEAP-VO on MPI Sintel shows over 60% gains in ATE, RPE trans, and RPE rot, validating robustness and effectiveness. The results indicate MONA's potential to improve markerless dataset generation and urban-planning workflows by enabling more accurate camera trajectory estimation in the presence of moving objects.

Abstract

Dynamic urban environments, characterized by moving cameras and objects, pose significant challenges for camera trajectory estimation by complicating the distinction between camera-induced and object motion. We introduce MONA, a novel framework designed for robust moving object detection and segmentation from videos shot by dynamic cameras. MONA comprises two key modules: Dynamic Points Extraction, which leverages optical flow and tracking any point to identify dynamic points, and Moving Object Segmentation, which employs adaptive bounding box filtering, and the Segment Anything for precise moving object segmentation. We validate MONA by integrating with the camera trajectory estimation method LEAP-VO, and it achieves state-of-the-art results on the MPI Sintel dataset comparing to existing methods. These results demonstrate MONA's effectiveness for moving object detection and its potential in many other applications in the urban planning field.

Figures (3)

• Figure 1: The overall pipeline of MONA. Our method consists of two main modules: Dynamic Points Extraction and Moving Object Segmentation. In Dynamic Points Extraction, random points are selected and LEAP-VO estimates the probability of each point being dynamic. RAFT then computes the optical flow, and a post-processing algorithm combines these results to identify dynamic points. In Moving Object Segmentation, YOLO detects all object bounding boxes, which are filtered using the dynamic points through our designed adaptive bounding box filter to identify moving objects. Finally, the filtered bounding boxes are used as prompt input to SAM to get the segmentation of the moving objects from the input video.
• Figure 2: Qualitative Comparison on Estimated Camera Trajectory between different methods. We run original LEAP-VO and LEAP-VO with MONA on MPI Sintel Dataset butler2012sintel. Two estimated trajectories are selected as the comparison. The MONA+LEAP-VO provides a more accurate estimated trajectory as it is more similar to the GT, which demonstrates the effectiveness of our method in camera trajectory estimation tasks.
• Figure 3: Qualitative Ablation Study of MONA. We visualize and compare the quality of masks produced by different prompts input to SAM: (b) pure dynamic points (c) pure bounding boxes (without filtering), and (d) bounding boxes with dynamic points filtering strategy.