HieraRS: A Hierarchical Segmentation Paradigm for Remote Sensing Enabling Multi-Granularity Interpretation and Cross-Domain Transfer

Abstract

Hierarchical land cover and land use (LCLU) classification aims to assign pixel-wise labels with multiple levels of semantic granularity to remote sensing (RS) imagery. However, existing deep learning-based methods face two major challenges: 1) They predominantly adopt a flat classification paradigm, which limits their ability to generate end-to-end multi-granularity hierarchical predictions aligned with tree-structured hierarchies used in practice. 2) Most cross-domain studies focus on performance degradation caused by sensor or scene variations, with limited attention to transferring LCLU models to cross-domain tasks with heterogeneous hierarchies (e.g., LCLU to crop classification). These limitations hinder the flexibility and generalization of LCLU models in practical applications. To address these challenges, we propose HieraRS, a novel hierarchical interpretation paradigm that enables multi-granularity predictions and supports the efficient transfer of LCLU models to cross-domain tasks with heterogeneous tree-structured hierarchies. We introduce the Bidirectional Hierarchical Consistency Constraint Mechanism (BHCCM), which can be seamlessly integrated into mainstream flat classification models to generate hierarchical predictions, while improving both semantic consistency and classification accuracy. Furthermore, we present TransLU, a dual-branch cross-domain transfer framework comprising two key components: Cross-Domain Knowledge Sharing (CDKS) and Cross-Domain Semantic Alignment (CDSA). TransLU supports dynamic category expansion and facilitates the effective adaptation of LCLU models to heterogeneous hierarchies. In addition, we construct MM-5B, a large-scale multi-modal hierarchical land use dataset featuring pixel-wise annotations. The code and MM-5B dataset will be released at: https://github.com/AI-Tianlong/HieraRS.

Figures (12)

• Figure 1: Illustration of two challenges. (a) Semantic inconsistency between flat classification paradigms and multi-granularity hierarchical requirements. (b) Limited generalization of models with fixed hierarchies when facing dynamic category expansion and cross-domain transfer.
• Figure 2: Distribution of acquisition time differences between the primary GaoFen-2 imagery and the supplementary Sentinel-2 and Google Earth sources in the MM-5B dataset.
• Figure 3: Illustration of the MM-5B dataset. The left panel shows visualizations of three paired data sources and their corresponding labels at three hierarchical levels: Google Earth (RGB), GaoFen-2 (B4-B3-B2, false color), and Sentinel-2 (B12-B8A-B4, SWIR). The right panel presents the tree-structured hierarchical classification system of the MM-5B, consisting of three levels: $L1$ with 4 categories, $L2$ with 9 categories, and $L3$ with 18 categories.
• Figure 4: Overview of HieraRS. Stage i: (a) BHCCM seamlessly integrates into flat semantic segmentation baselines and converts them into hierarchical LCLU classification models. Stage ii: (b) TransLU enables cross-domain transfer of the Stage-i model under heterogeneous label systems. (c) CDKS performs feature interaction and knowledge sharing across domains; (d) CDSA aligns feature semantics across domains; and (e) shows the cross-domain tree-structured hierarchy used to transfer the hierarchical LCLU model to crop classification.
• Figure 5: Detailed structure of BHCCM. The figure illustrates the bidirectional information flow pathways, where semantic feature interaction and fusion across hierarchical levels are achieved via coarse-to-fine (green arrows) and fine-to-coarse (blue arrows) connections.