Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A class-driven hierarchical ResNet for classification of multispectral remote sensing images

Giulio Weikmann, Gianmarco Perantoni, Lorenzo Bruzzone

TL;DR

This paper tackles land-cover classification from multispectral time-series under limited labeled data by introducing a class-driven hierarchical ResNet that explicitly models macro, intermediate, and micro class levels. It augments a ResNet backbone with multiple classification branches and a hierarchy penalty that enforces coherent hierarchical transitions, enabling more discriminative features at different semantic depths. Evaluated on two Amazon Sentinel-2 tiles reflecting real-world imbalanced data, the approach yields improved F1 scores for minority classes and better hierarchical consistency, while maintaining competitive overall accuracy during fine-tuning to new areas. The architecture is modular and adaptable as a backbone for new tasks with limited annotations, making it practical for scalable, domain-specific RS classification. The work demonstrates strong potential for hierarchical RS analysis and domain adaptation with limited training data.

Abstract

This work presents a multitemporal class-driven hierarchical Residual Neural Network (ResNet) designed for modelling the classification of Time Series (TS) of multispectral images at different semantical class levels. The architecture consists of a modification of the ResNet where we introduce additional branches to perform the classification at the different hierarchy levels and leverage on hierarchy-penalty maps to discourage incoherent hierarchical transitions within the classification. In this way, we improve the discrimination capabilities of classes at different levels of semantic details and train a modular architecture that can be used as a backbone network for introducing new specific classes and additional tasks considering limited training samples available. We exploit the class-hierarchy labels to train efficiently the different layers of the architecture, allowing the first layers to train faster on the first levels of the hierarchy modeling general classes (i.e., the macro-classes) and the intermediate classes, while using the last ones to discriminate more specific classes (i.e., the micro-classes). In this way, the targets are constrained in following the hierarchy defined, improving the classification of classes at the most detailed level. The proposed modular network has intrinsic adaptation capability that can be obtained through fine tuning. The experimental results, obtained on two tiles of the Amazonian Forest on 12 monthly composites of Sentinel 2 images acquired during 2019, demonstrate the effectiveness of the hierarchical approach in both generalizing over different hierarchical levels and learning discriminant features for an accurate classification at the micro-class level on a new target area, with a better representation of the minoritarian classes.

A class-driven hierarchical ResNet for classification of multispectral remote sensing images

TL;DR

This paper tackles land-cover classification from multispectral time-series under limited labeled data by introducing a class-driven hierarchical ResNet that explicitly models macro, intermediate, and micro class levels. It augments a ResNet backbone with multiple classification branches and a hierarchy penalty that enforces coherent hierarchical transitions, enabling more discriminative features at different semantic depths. Evaluated on two Amazon Sentinel-2 tiles reflecting real-world imbalanced data, the approach yields improved F1 scores for minority classes and better hierarchical consistency, while maintaining competitive overall accuracy during fine-tuning to new areas. The architecture is modular and adaptable as a backbone for new tasks with limited annotations, making it practical for scalable, domain-specific RS classification. The work demonstrates strong potential for hierarchical RS analysis and domain adaptation with limited training data.

Abstract

This work presents a multitemporal class-driven hierarchical Residual Neural Network (ResNet) designed for modelling the classification of Time Series (TS) of multispectral images at different semantical class levels. The architecture consists of a modification of the ResNet where we introduce additional branches to perform the classification at the different hierarchy levels and leverage on hierarchy-penalty maps to discourage incoherent hierarchical transitions within the classification. In this way, we improve the discrimination capabilities of classes at different levels of semantic details and train a modular architecture that can be used as a backbone network for introducing new specific classes and additional tasks considering limited training samples available. We exploit the class-hierarchy labels to train efficiently the different layers of the architecture, allowing the first layers to train faster on the first levels of the hierarchy modeling general classes (i.e., the macro-classes) and the intermediate classes, while using the last ones to discriminate more specific classes (i.e., the micro-classes). In this way, the targets are constrained in following the hierarchy defined, improving the classification of classes at the most detailed level. The proposed modular network has intrinsic adaptation capability that can be obtained through fine tuning. The experimental results, obtained on two tiles of the Amazonian Forest on 12 monthly composites of Sentinel 2 images acquired during 2019, demonstrate the effectiveness of the hierarchical approach in both generalizing over different hierarchical levels and learning discriminant features for an accurate classification at the micro-class level on a new target area, with a better representation of the minoritarian classes.

Paper Structure

This paper contains 11 sections, 4 equations, 2 figures, 4 tables.

Table of Contents

  1. INTRODUCTION
  2. Methodology
  3. Class-driven Hierarchical ResNet
  4. Hierarchy Penalty Term
  5. Modularity and Fine-Tuning Strategy
  6. Dataset Description
  7. Design of the experiments
  8. Training Phase
  9. Fine-Tuning
  10. Experimental results
  11. Conclusion

Figures (2)

  • Figure 1: Architecture of the proposed Class-driven Hierarchical ResNet.
  • Figure 2: Qualitative example of the classification maps at the micro-class level obtained on a portion of the "21KUQ" tile: (a) reference monthly composite of July 2019, (b) reference HRLC test set, (c) classification obtained using the Std. ResNet and (d) classification obtained considering the proposed hierarchical class-driven ResNet.