Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging knowledge distillation for partial multi-task learning from multiple remote sensing datasets

Hoàng-Ân Lê, Minh-Tan Pham

TL;DR

The paper addresses learning multiple remote-sensing tasks when training data are annotated for only one task per example, by leveraging knowledge distillation from a frozen teacher. It extends partial multi-task learning by providing both soft targets and feature-level supervision for unannotated tasks, enabling better joint representations without ground truth. In ISPRS-based experiments, soft-label distillation and PDF-Distil feature distillation yield strong improvements for both object detection and semantic segmentation. The work demonstrates a data-efficient path to fuse heterogeneous remote-sensing annotations, reducing parameter needs while enhancing accuracy.

Abstract

Partial multi-task learning where training examples are annotated for one of the target tasks is a promising idea in remote sensing as it allows combining datasets annotated for different tasks and predicting more tasks with fewer network parameters. The naïve approach to partial multi-task learning is sub-optimal due to the lack of all-task annotations for learning joint representations. This paper proposes using knowledge distillation to replace the need of ground truths for the alternate task and enhance the performance of such approach. Experiments conducted on the public ISPRS 2D Semantic Labeling Contest dataset show the effectiveness of the proposed idea on partial multi-task learning for semantic tasks including object detection and semantic segmentation in aerial images.

Leveraging knowledge distillation for partial multi-task learning from multiple remote sensing datasets

TL;DR

The paper addresses learning multiple remote-sensing tasks when training data are annotated for only one task per example, by leveraging knowledge distillation from a frozen teacher. It extends partial multi-task learning by providing both soft targets and feature-level supervision for unannotated tasks, enabling better joint representations without ground truth. In ISPRS-based experiments, soft-label distillation and PDF-Distil feature distillation yield strong improvements for both object detection and semantic segmentation. The work demonstrates a data-efficient path to fuse heterogeneous remote-sensing annotations, reducing parameter needs while enhancing accuracy.

Abstract

Partial multi-task learning where training examples are annotated for one of the target tasks is a promising idea in remote sensing as it allows combining datasets annotated for different tasks and predicting more tasks with fewer network parameters. The naïve approach to partial multi-task learning is sub-optimal due to the lack of all-task annotations for learning joint representations. This paper proposes using knowledge distillation to replace the need of ground truths for the alternate task and enhance the performance of such approach. Experiments conducted on the public ISPRS 2D Semantic Labeling Contest dataset show the effectiveness of the proposed idea on partial multi-task learning for semantic tasks including object detection and semantic segmentation in aerial images.
Paper Structure (6 sections, 2 figures, 2 tables)

This paper contains 6 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Experiments
  4. Dataset and setup
  5. Results
  6. Conclusion

Figures (2)

  • Figure 1: Multi-task partially supervised learning with knowledge distillation. The vanilla setup is shown in the yellow box including a shared encoder and two task heads for object detection (in red) and semantic segmentation (in green with their respective supervised losses. We illustrate the data flow for the iteration with annotated detection using solid red lines while the dotted green lines are for annotated segmentation in the next iteration. The teacher networks (in gray) provide soft labels and/or feature loss to train the task head without annotations (illustrated for segmentation).
  • Figure 2: A sample of Vaihingen (left) and Postdam (right) subsets isprs2012. Vaihingen data compose of IR-R-G channels thus appear with reddish vegetation while Potsdam is with regular R-G-B imagery. The lower ground sampling distance of Vaihingen results in smaller and more objects in the same image crop compared to Potsdam.