A Unified Model Selection Technique for Spectral Clustering Based Motion Segmentation
Yuxiang Huang, John Zelek
TL;DR
The paper tackles automatic determination of the number of motion groups in spectral clustering-based motion segmentation under challenging dynamic scenes. It introduces a unified model selection framework that combines four criteria—Silhouette score, eigengap, Davies-Bouldin index, and Calinski-Harabasz index—to infer the optimal number of motion clusters by averaging criterion confidences across a range of candidate counts, including handling fused affinity matrices. The approach leverages a baseline motion segmentation pipeline with two motion cues and co-regularized multi-view spectral clustering, and demonstrates competitive performance on the KT3DMoSeg dataset, approaching a baseline that uses ground-truth motion counts. This work simplifies practical deployment by automating model selection and showing robust segmentation across varying scene complexities.
Abstract
Motion segmentation is a fundamental problem in computer vision and is crucial in various applications such as robotics, autonomous driving and action recognition. Recently, spectral clustering based methods have shown impressive results on motion segmentation in dynamic environments. These methods perform spectral clustering on motion affinity matrices to cluster objects or point trajectories in the scene into different motion groups. However, existing methods often need the number of motions present in the scene to be known, which significantly reduces their practicality. In this paper, we propose a unified model selection technique to automatically infer the number of motion groups for spectral clustering based motion segmentation methods by combining different existing model selection techniques together. We evaluate our method on the KT3DMoSeg dataset and achieve competitve results comparing to the baseline where the number of clusters is given as ground truth information.