SAModified: A Foundation Model-Based Zero-Shot Approach for Refining Noisy Land-Use Land-Cover Maps

TL;DR

Noisy ground truth in LULC datasets hampers accurate mapping. The authors propose a two-stage, zero-shot SAM-based method that first delineates land parcels and then relabels pixels by the dominant class within each parcel. This denoising yields substantial improvements in downstream segmentation performance and outperforms traditional unsupervised baselines like K-means and DBSCAN. The approach enhances reliability and scalability of large-scale LULC mapping for remote sensing applications, with potential extension to broader EO foundation models.

Abstract

Land-use and land cover (LULC) analysis is critical in remote sensing, with wide-ranging applications across diverse fields such as agriculture, utilities, and urban planning. However, automating LULC map generation using machine learning is rendered challenging due to noisy labels. Typically, the ground truths (e.g. ESRI LULC, MapBioMass) have noisy labels that hamper the model's ability to learn to accurately classify the pixels. Further, these erroneous labels can significantly distort the performance metrics of a model, leading to misleading evaluations. Traditionally, the ambiguous labels are rectified using unsupervised algorithms. These algorithms struggle not only with scalability but also with generalization across different geographies. To overcome these challenges, we propose a zero-shot approach using the foundation model, Segment Anything Model (SAM), to automatically delineate different land parcels/regions and leverage them to relabel the unsure pixels by using the local label statistics within each detected region. We achieve a significant reduction in label noise and an improvement in the performance of the downstream segmentation model by $\approx 5\%$ when trained with denoised labels.

Figures (5)

• Figure 1: Graphical Abstract. (a) depicts the noisy ground truth annotations, which contain incorrect and ambiguous labels. (b) shows the denoised ground truth after applying the proposed zero-shot approach using the Segment Anything Model (SAM), resulting in cleaner and more reliable labels. The zoomed-in regions highlight the improvements in label accuracy achieved by the method.
• Figure 2: Overview of the Proposed Two-Stage Approach. In Stage 1, the Segment Anything Model (SAM) is employed to delineate distinct land parcels in the input imagery using zero-shot learning. Stage 2 involves analyzing the local label statistics within each identified parcel and reassigning labels based on the dominant class within each region.
• Figure 3: Reduction of label noise in stray pixels. (a) HLS input image, (b) Noisy ground truth with uncertain pixels labelled as 'mosaic of uses' (yellow), and (c) Denoised ground truth with these pixels reassigned to the forest class (dark green) based on SAM's segmentation.
• Figure 4: Improvements in class boundaries. (a) HLS input image, (b) Noisy ground truth with uncertain pixels (yellow), and (c) Denoised ground truth with more accurate boundaries between classes based on SAM's segmentation.
• Figure 5: Qualitative Comparison. Comparison of different methods for cleaning up noisy labels. The first column shows the input image. The middle column (blue box) highlights clusters or segments identified by KMeans (BL1), DBSCAN (BL2), and the proposed approach. The right column (red box) displays the corresponding denoised labels after majority voting within each cluster or segment. The proposed method (third row) using SAM outperforms the traditional clustering methods.