Hyperbolic Active Learning for Semantic Segmentation under Domain Shift

TL;DR

HALO introduces a novel pixel-level active learning strategy for semantic segmentation under domain shift by interpreting the hyperbolic radius in the Poincaré ball as a data-scarcity indicator that, when combined with prediction entropy, approximates epistemic uncertainty. The method forms a per-pixel acquisition map A = R ⊙ H and achieves state-of-the-art results across ADA benchmarks GTA→Cityscapes, SYNTHIA→Cityscapes, and Cityscapes→ACDC, even surpassing supervised domain adaptation with modest labeling budgets (as low as 5%). A key contribution is Hyperbolic Feature Reweighting (HFR), which stabilizes training in hyperbolic space and improves robustness in ADA scenarios. The work demonstrates that hyperbolic representations, when paired with entropy signals, effectively guide label acquisition, offering practical gains in label efficiency and enabling competitive domain adaptation with minimal supervision.

Abstract

We introduce a hyperbolic neural network approach to pixel-level active learning for semantic segmentation. Analysis of the data statistics leads to a novel interpretation of the hyperbolic radius as an indicator of data scarcity. In HALO (Hyperbolic Active Learning Optimization), for the first time, we propose the use of epistemic uncertainty as a data acquisition strategy, following the intuition of selecting data points that are the least known. The hyperbolic radius, complemented by the widely-adopted prediction entropy, effectively approximates epistemic uncertainty. We perform extensive experimental analysis based on two established synthetic-to-real benchmarks, i.e. GTAV $\rightarrow$ Cityscapes and SYNTHIA $\rightarrow$ Cityscapes. Additionally, we test HALO on Cityscape $\rightarrow$ ACDC for domain adaptation under adverse weather conditions, and we benchmark both convolutional and attention-based backbones. HALO sets a new state-of-the-art in active learning for semantic segmentation under domain shift and it is the first active learning approach that surpasses the performance of supervised domain adaptation while using only a small portion of labels (i.e., 1%).

Figures (10)

• Figure 1: Overview of HALO. Pixels are encoded into the hyperbolic Poincaré ball and classified in the pseudo-label $\mathcal{\hat{Y}}$. The hyperbolic radius of the pixel embeddings defines the new hyperbolic score map $\mathcal{R}$. The prediction entropy $\mathcal{H}$ is extracted as the entropy of the softmax probabilities. Combining $\mathcal{R}$ and $\mathcal{H}$ we define the data acquisition score map $\mathcal{A}$, which is used to query new labels $\mathcal{Y}$.
• Figure 2: (left) Plot of class average radius vs. the percentage of total pixels in the target dataset; (center) Plot of the class average entropy vs. class accuracy; (right) Plot of labeling budget vs. correlation between class average radius and percentage of total pixels (blue) and between class average entropy and class accuracy (orange).
• Figure 3: Plot of the class average radius vs. class accuracy.
• Figure 4: (a) Original image; (b) Radius map depicting the hyperbolic radii of pixel embeddings; (c) Pixels (yellow) that have been selected for data acquisition. See Sec. \ref{['sec:method']} for details; (d) HALO prediction; (e) Ground Truth annotations. Zoom in for the details.
• Figure 5: (left) Performance on GTAV $\rightarrow$ Cityscapes with different budgets. (right) Evolution of the variance (y axis) of selected pixels distributions with varying budget (x axis).