MarsSeg: Mars Surface Semantic Segmentation with Multi-level Extractor and Connector

TL;DR

MarsSeg tackles the challenging problem of semantic segmentation on Martian terrain, where inter-class differences are subtle and labeled data are limited. It introduces an encoder-decoder backbone with reduced down-sampling and a feature enhancement bridge comprising Mini-ASPP, Polarized Self-Attention (PSA), and the Strip Pyramid Pooling Module (SPPM) to capture both fine local details and high-level semantics, connected via a Focal-Dice–based adaptive loss. The loss incorporates adaptive class weights $w_k = 1/( ext{IoU}_k + \alpha)$ and deep supervision across decoder layers, enabling robust performance under severe class imbalance. Experiments on Mars-Seg and AI4Mars show state-of-the-art results, validating the effectiveness of the three enhancement components and the proposed loss for accurate and robust Mars surface segmentation.

Abstract

The segmentation and interpretation of the Martian surface play a pivotal role in Mars exploration, providing essential data for the trajectory planning and obstacle avoidance of rovers. However, the complex topography, similar surface features, and the lack of extensive annotated data pose significant challenges to the high-precision semantic segmentation of the Martian surface. To address these challenges, we propose a novel encoder-decoder based Mars segmentation network, termed MarsSeg. Specifically, we employ an encoder-decoder structure with a minimized number of down-sampling layers to preserve local details. To facilitate a high-level semantic understanding across the shadow multi-level feature maps, we introduce a feature enhancement connection layer situated between the encoder and decoder. This layer incorporates Mini Atrous Spatial Pyramid Pooling (Mini-ASPP), Polarized Self-Attention (PSA), and Strip Pyramid Pooling Module (SPPM). The Mini-ASPP and PSA are specifically designed for shadow feature enhancement, thereby enabling the expression of local details and small objects. Conversely, the SPPM is employed for deep feature enhancement, facilitating the extraction of high-level semantic category-related information. Experimental results derived from the Mars-Seg and AI4Mars datasets substantiate that the proposed MarsSeg outperforms other state-of-the-art methods in segmentation performance, validating the efficacy of each proposed component.

Figures (9)

• Figure 1: The MarsSeg framework primarily comprises three integral components: an encoder, feature enhancement connections, and a decoder. The feature enhancement connections are further subdivided into the Mini-ASPP, PSA, and SPPM. The Mini-ASPP and PSA modules collectively form the enhancement connections for shallow features, whereas the SPPM is responsible for the enhancement connections for mid-level features. These enhancement connections are effectively adept at adapting to the complex and variable Martian terrain, as well as the minimal inter-class feature differences.
• Figure 2: The Mini-ASPP is characterized by three distinct atrous convolution layers, each employing varying rates. This unique configuration facilitates the acquisition of local multi-scale representations efficiently.
• Figure 3: The overview of the PSA. PSA executes a lightweight self-attention process from both the channel and spatial perspectives.
• Figure 4: Some representative examples of Mars' landforms.
• Figure 5: The overview of the strip pooling pyramid module (SPPM).