Predicting Road Surface Anomalies by Visual Tracking of a Preceding Vehicle

TL;DR

The paper tackles predictive detection of road surface anomalies by tracking the vertical motion of a preceding vehicle using an ego camera, exploiting the signal $y(t)$ and compensating for ego-motion with the pitch estimate $\hat{\varphi}(t)$. The approach combines Mask2Former-based vehicle detection, CoTracker-based tracking, and a robust pitch-estimation method using a fundamental matrix $F(\varphi)$ to produce a compensated trajectory $\hat{y}_c(t)$, from which a short-window standard deviation $s(t)$ signals anomalies. It introduces the SVAR dataset with controlled and in-the-wild subsets to validate the method, and demonstrates that pitch compensation significantly improves detection accuracy and reduces false positives compared with baselines such as uncompensated tracking and RANSAC-based essential matrix estimation. The solution operates in real time on consumer hardware and offers a practical mechanism for early warnings and chassis-control adjustments in autonomous or assisted driving, particularly under poor visibility or occlusions.

Abstract

A novel approach to detect road surface anomalies by visual tracking of a preceding vehicle is proposed. The method is versatile, predicting any kind of road anomalies, such as potholes, bumps, debris, etc., unlike direct observation methods that rely on training visual detectors of those cases. The method operates in low visibility conditions or in dense traffic where the anomaly is occluded by a preceding vehicle. Anomalies are detected predictively, i.e., before a vehicle encounters them, which allows to pre-configure low-level vehicle systems (such as chassis) or to plan an avoidance maneuver in case of autonomous driving. A challenge is that the signal coming from camera-based tracking of a preceding vehicle may be weak and disturbed by camera ego motion due to vibrations affecting the ego vehicle. Therefore, we propose an efficient method to compensate camera pitch rotation by an iterative robust estimator. Our experiments on both controlled setup and normal traffic conditions show that road anomalies can be detected reliably at a distance even in challenging cases where the ego vehicle traverses imperfect road surfaces. The method is effective and performs in real time on standard consumer hardware.

Figures (7)

• Figure 1: The main idea is to track the motion of a preceding vehicle using a camera mounted in the ego vehicle to detect anomalies on the road surface, such as potholes, bumps, etc. The vertical trajectory $y(t)$ measured by the camera reflects the surface profile beneath the preceding vehicle. Anomalies in this signal, are anomalies of the road surface. A challenge is that the camera of the ego vehicle moves. The translation motion is not important, but the camera pitch rotation $\varphi(t)$, caused by other uneveness of the road, distorts the signal. We visually estimate the ego rotation of the camera $\varphi(t)$ and compensate for it, which results in improved accuracy of the method.
• Figure 2: Flowchart of the proposed method. The vertical trajectory of the preceding vehicle is tracked by instance segmentation and CoTracker karaev2023cotracker. The pitch is estimated from correspondences on static background features, and the egomotion is compensated. Finally, the resulting visual response to the road profile is calculated as standard deviation of the compensated trajectory in a short window (\ref{['eq:std']}).
• Figure 3: Examples of anomalies, sources of high-vertical acceleration, on the road surface in our SVAR dataset, controlled and int-the-wild subsets. Note that except for the leftmost column, the road surface anomalies are not visible to a human observer, but are detectable by observing the preceding vehicle. For better illustration, see supplementary videos at https://cmp.felk.cvut.cz/%7Ecechj/video/iv-2025/ .
• Figure 4: Visual responses $s(t)$ estimated by Eq. (\ref{['eq:std']}) vs. car distances for the controlled experiment. Plot shows positive class samples (speed bumps), negative class samples (natural background), and the theoretical expectation given by our model in Eq. (\ref{['eq:theory']}).
• Figure 5: Prediction of pitch angle of an ego vehicle traversing a speed bump (top). Corresponding trajectory of the preceding car with the egomotion compensated using the estimated pitch (bottom).