BiCoR-Seg: Bidirectional Co-Refinement Framework for High-Resolution Remote Sensing Image Segmentation

TL;DR

BiCoR-Seg tackles high intra-class variability and inter-class similarity in high-resolution remote sensing segmentation by introducing a heatmap-driven bidirectional co-refinement framework. The key idea is to couple pixel-level features with dynamic, image-adaptive class embeddings through the Heatmap-driven Information Synergy (HBIS) module, which comprises a feature-to-class update, a class-to-feature modulation, and class heatmaps that guide updates via top-K region pooling. Hierarchical heatmap supervision and a Fisher discriminant loss jointly enforce spatial localization and semantic separability across decoder stages, improving both boundary sharpness and class discrimination. Extensive experiments on LoveDA, Vaihingen, and Potsdam demonstrate state-of-the-art performance and strong interpretability, with ablations confirming the effectiveness of HBIS and the auxiliary losses for robust generalization.

Abstract

High-resolution remote sensing image semantic segmentation (HRSS) is a fundamental yet critical task in the field of Earth observation. However, it has long faced the challenges of high inter-class similarity and large intra-class variability. Existing approaches often struggle to effectively inject abstract yet strongly discriminative semantic knowledge into pixel-level feature learning, leading to blurred boundaries and class confusion in complex scenes. To address these challenges, we propose Bidirectional Co-Refinement Framework for HRSS (BiCoR-Seg). Specifically, we design a Heatmap-driven Bidirectional Information Synergy Module (HBIS), which establishes a bidirectional information flow between feature maps and class embeddings by generating class-level heatmaps. Based on HBIS, we further introduce a hierarchical supervision strategy, where the interpretable heatmaps generated by each HBIS module are directly utilized as low-resolution segmentation predictions for supervision, thereby enhancing the discriminative capacity of shallow features. In addition, to further improve the discriminability of the embedding representations, we propose a cross-layer class embedding Fisher Discriminative Loss to enforce intra-class compactness and enlarge inter-class separability. Extensive experiments on the LoveDA, Vaihingen, and Potsdam datasets demonstrate that BiCoR-Seg achieves outstanding segmentation performance while offering stronger interpretability. The released code is available at https://github.com/ShiJinghao566/BiCoR-Seg.

Figures (5)

• Figure 1: Overall architecture of the proposed BiCoR-Seg framework. The proposed framework consists of an encoder for multi-scale feature extraction, a decoder composed of multiple cascaded HBIS modules, and an output head.
• Figure 2: Architecture of the proposed HBIS module. The HBIS module is composed of two information pathways: Feature-to-Class Embedding (F2CE) and Class Embedding-to-Feature (CE2F). Three types of heatmaps are utilized in the bidirectional interaction: the raw class heatmap, the normalized heatmap $\tilde{H}$ (with a BN layer for normalization), and the Softmax-normalized heatmap $H^{soft}$ for semantic fusion.
• Figure 3: Effect of the Top-K threshold on model performance on the LoveDA dataset.
• Figure 4: Qualitative comparison of segmentation results on challenging remote sensing scenes. BiCoR-Seg produces more complete and semantically consistent segmentation maps compared with other representative methods.
• Figure 5: Visualization of class heatmaps generated by BiCoR-Seg for representative categories. The HBIS module effectively highlights category-specific regions while suppressing irrelevant background noise.