OFFSEG: A Semantic Segmentation Framework For Off-Road Driving

TL;DR

OFFSEG tackles off-road semantic segmentation where uneven terrain and unstructured boundaries hinder typical urban datasets. It combines a four-class semantic segmentation stage with a color-based sub-class refinement within the traversable region, using BiSeNetV2 or HRNETV2+OCR backbones and a MobileNetV2 classifier trained via transfer learning. Evaluations on RELLIS-3D, RUGD, and IISERB campus show strong mIoU scores and rich sub-class labeling, with feasible on-device speed on Jetson hardware. The approach mitigates class imbalance and enhances perceptual richness, facilitating safer and more informed autonomous off-road navigation.

Abstract

Off-road image semantic segmentation is challenging due to the presence of uneven terrains, unstructured class boundaries, irregular features and strong textures. These aspects affect the perception of the vehicle from which the information is used for path planning. Current off-road datasets exhibit difficulties like class imbalance and understanding of varying environmental topography. To overcome these issues we propose a framework for off-road semantic segmentation called as OFFSEG that involves (i) a pooled class semantic segmentation with four classes (sky, traversable region, non-traversable region and obstacle) using state-of-the-art deep learning architectures (ii) a colour segmentation methodology to segment out specific sub-classes (grass, puddle, dirt, gravel, etc.) from the traversable region for better scene understanding. The evaluation of the framework is carried out on two off-road driving datasets, namely, RELLIS-3D and RUGD. We have also tested proposed framework in IISERB campus frames. The results show that OFFSEG achieves good performance and also provides detailed information on the traversable region.

Figures (8)

• Figure 1: Segmentation results on IISERB campus frames (top left) Raw image taken from the vehicle (top right) Semantic segmentation of four clases using BiSeNetV2 (below) Color segmentation on the traversable region providing additional information about the traversable region
• Figure 2: OFFSEG consists of two stages. First, pooling of different classes into four and performing semantic image segmentation. Second, the RoI (region of interest) obtained from the segmentation passes through the color segmentation algorithm which segments and classifies sub classes like grass, mud, puddle, etc and append them as a final output.
• Figure 3: Segmentation results from OFFSEG for four class model have been compared with the HRNET 20 class model
• Figure 4: Color segmentation results on RELLIS-3D (a) Traversable class obtained as RoI from segmentation (b) grass, (c) puddle, (d) mud obtained from color segmentation from RoI.
• Figure 5: Color segmentation results on RUGD (a) Traversable class obtained as RoI from segmentation (b) mulch, (c) gravel obtained from color segmentation from RoI.