Monocular Per-Object Distance Estimation with Masked Object Modeling

TL;DR

This work tackles per-object distance estimation from monocular imagery and introduces DistFormer with a self-supervised Masked Object Modeling (MoM) objective that reconstructs object-region content under masking. MoM is trained in a single unified stage with the distance regression loss, yielding strong regularization and improved generalization, including zero-shot and few-shot transfer and occlusion robustness. The approach achieves state-of-the-art or competitive results on KITTI, NuScenes, and MOTSynth, with extensive ablations and robustness analyses, and it highlights potential extensions to pose estimation, detection, and segmentation along with domain adaptation and lightweight variants. The work has practical implications for autonomous driving and surveillance, offering avenues for more robust monocular 3D reasoning while addressing privacy and detector-dependence considerations.

Abstract

Per-object distance estimation is critical in surveillance and autonomous driving, where safety is crucial. While existing methods rely on geometric or deep supervised features, only a few attempts have been made to leverage self-supervised learning. In this respect, our paper draws inspiration from Masked Image Modeling (MiM) and extends it to multi-object tasks. While MiM focuses on extracting global image-level representations, it struggles with individual objects within the image. This is detrimental for distance estimation, as objects far away correspond to negligible portions of the image. Conversely, our strategy, termed Masked Object Modeling (MoM), enables a novel application of masking techniques. In a few words, we devise an auxiliary objective that reconstructs the portions of the image pertaining to the objects detected in the scene. The training phase is performed in a single unified stage, simultaneously optimizing the masking objective and the downstream loss (i.e., distance estimation). We evaluate the effectiveness of MoM on a novel reference architecture (DistFormer) on the standard KITTI, NuScenes, and MOTSynth datasets. Our evaluation reveals that our framework surpasses the SoTA and highlights its robust regularization properties. The MoM strategy enhances both zero-shot and few-shot capabilities, from synthetic to real domain. Finally, it furthers the robustness of the model in the presence of occluded or poorly detected objects. Code is available at https://github.com/apanariello4/DistFormer

Figures (1)

• Figure A: RMSE and $\delta_{<1.25}$ at varying bounding box noise $r$.