A$^2$LC: Active and Automated Label Correction for Semantic Segmentation

TL;DR

A^2LC introduces a dual-source label correction framework for semantic segmentation that combines an automatic Label Correction Module with an adaptively balanced acquisition function to actively and efficiently correct mislabeled pixels. By cascading manual corrections with automatic propagation, and by prioritizing tail-class corrections through adaptive weighting, the method achieves substantial performance gains under reduced annotation budgets on Cityscapes and PASCAL VOC 2012. The approach yields significant improvements over prior active-label-correction methods, demonstrates strong synergy between LCM and human input, and offers practical cost savings for large-scale semantic segmentation tasks.

Abstract

Active Label Correction (ALC) has emerged as a promising solution to the high cost and error-prone nature of manual pixel-wise annotation in semantic segmentation, by actively identifying and correcting mislabeled data. Although recent work has improved correction efficiency by generating pseudo-labels using foundation models, substantial inefficiencies still remain. In this paper, we introduce A$^2$LC, an Active and Automated Label Correction framework for semantic segmentation, where manual and automatic correction stages operate in a cascaded manner. Specifically, the automatic correction stage leverages human feedback to extend label corrections beyond the queried samples, thereby maximizing cost efficiency. In addition, we introduce an adaptively balanced acquisition function that emphasizes underrepresented tail classes, working in strong synergy with the automatic correction stage. Extensive experiments on Cityscapes and PASCAL VOC 2012 demonstrate that A$^2$LC significantly outperforms previous state-of-the-art methods. Notably, A$^2$LC exhibits high efficiency by outperforming previous methods with only 20% of their budget, and shows strong effectiveness by achieving a 27.23% performance gain under the same budget on Cityscapes.

Figures (11)

• Figure 1: Effectiveness of A$^2$LC framework. Our model demonstrates clear effectiveness, saturating by round 6 on Cityscapes and round 5 on PASCAL. At these points, it outperforms the baseline by 11.81 and 4.44 mIoU and reaches 94% and 95% of full supervision.
• Figure 2: Overview of A$^2$LC framework. We execute Grounded SAM on unlabeled images to generate initial pseudo-labels. For each of the $R$ rounds, the model $\theta_{r-1}$ trained with pseudo-labels selects $B$ masks via the acquisition function to query for manual correction. Then, the model $\psi_r$ in Label Correction Module (LCM) is trained using the queried masks and corrects the labels of unqueried masks. After both manual and automatic corrections, the pseudo-labels and model are updated, completing a single round of the correction cycle.
• Figure 3: LCM pipeline. LCM operates in two sequential steps: the model is first trained with accurately labeled masks queried in the current round, followed by correcting potentially mislabeled masks based on the model’s predictions. Here, a traffic light (orange) mask, mislabeled as a traffic sign (yellow), was corrected by the annotator. This manually corrected mask is then used to train the model, enabling automatic refinement of similar cases. As a result, three additional similar masks were automatically corrected.
• Figure 4: Visualization of automatically corrected masks for the car and train classes.
• Figure 5: Class distribution of sampled data. The x-axis shows classes sorted by pseudo-label frequency, and the y-axis shows the number of sampled masks. Unlike the baseline, which largely concentrates on head classes, our proposed ABC acquisition function substantially increases the sampling of tail classes