OT-Drive: Out-of-Distribution Off-Road Traversable Area Segmentation via Optimal Transport

TL;DR

This paper targets robust traversable area segmentation under out-of-distribution conditions by introducing OT-Drive, a two-component framework consisting of Scene Anchor Generator (SAG) and an Optimal Transport–based fusion module (OT Fusion). SAG disentangles scene variations into weather, time-of-day, and road type to construct semantic anchors that generalize to unseen conditions, while OT Fusion transports RGB and surface normal features onto the learned anchor manifold, enabling distribution-level, domain-invariant fusion. The method achieves state-of-the-art OOD generalization on ORFD (mIoU 95.16%) and strong cross-dataset transfer (mIoU 89.79%), with substantial improvements over baselines and real-time performance (≈21 FPS). These results demonstrate the practicality of distribution-level fusion guided by language-informed scene anchors for robust off-road perception with limited training data, and point toward open-world extensions with multimodal large language models.

Abstract

Reliable traversable area segmentation in unstructured environments is critical for planning and decision-making in autonomous driving. However, existing data-driven approaches often suffer from degraded segmentation performance in out-of-distribution (OOD) scenarios, consequently impairing downstream driving tasks. To address this issue, we propose OT-Drive, an Optimal Transport--driven multi-modal fusion framework. The proposed method formulates RGB and surface normal fusion as a distribution transport problem. Specifically, we design a novel Scene Anchor Generator (SAG) to decompose scene information into the joint distribution of weather, time-of-day, and road type, thereby constructing semantic anchors that can generalize to unseen scenarios. Subsequently, we design an innovative Optimal Transport-based multi-modal fusion module (OT Fusion) to transport RGB and surface normal features onto the manifold defined by the semantic anchors, enabling robust traversable area segmentation under OOD scenarios. Experimental results demonstrate that our method achieves 95.16% mIoU on ORFD OOD scenarios, outperforming prior methods by 6.35%, and 89.79% mIoU on cross-dataset transfer tasks, surpassing baselines by 13.99%.These results indicate that the proposed model can attain strong OOD generalization with only limited training data, substantially enhancing its practicality and efficiency for real-world deployment.

Figures (8)

• Figure 1: Existing segmentation methods exhibit poor OOD generalization. Red and green regions indicate false and true traversable areas, respectively.
• Figure 2: Overall architecture of the proposed OT-Drive. The framework is composed of two modules designed for OOD generalization in traversable area segmentation: 1) The Scene Anchor Generator (SAG) constructs scene-specific semantic anchors from the input image. 2) The OT Fusion Module leverages optimal transport to align RGB and surface normal features onto the manifold spanned by the semantic anchors, achieving distribution-level feature fusion.
• Figure 3: Scene Anchor Generator (SAG). Top: Scene attributes (weather, time-of-day, road type) are classified to derive the scene distribution. Bottom: Scene prototypes are weighted by the distribution to form the scene anchor.
• Figure 4: OT Fusion Module. The image and normal features are then transported to the scene anchor manifold via optimal transport for fusion.
• Figure 5: Long-tailed distribution analysis of the ORFD dataset.