Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ProtoFlow: Mitigating Forgetting in Class-Incremental Remote Sensing Segmentation via Low-Curvature Prototype Flow

Jiekai Wu, Rong Fu, Chuangqi Li, Zijian Zhang, Guangxin Wu, Hao Zhang, Shiyin Lin, Jianyuan Ni, Yang Li, Dongxu Zhang, Amir H. Gandomi, Simon Fong, Pengbin Feng

Abstract

Remote sensing segmentation in real deployment is inherently continual: new semantic categories emerge, and acquisition conditions shift across seasons, cities, and sensors. Despite recent progress, many incremental approaches still treat training steps as isolated updates, which leaves representation drift and forgetting insufficiently controlled. We present ProtoFlow, a time-aware prototype dynamics framework that models class prototypes as trajectories and learns their evolution with an explicit temporal vector field. By jointly enforcing low-curvature motion and inter-class separation, ProtoFlow stabilizes prototype geometry throughout incremental learning. Experiments on standard class- and domain-incremental remote sensing benchmarks show consistent gains over strong baselines, including up to 1.5-2.0 points improvement in mIoUall, together with reduced forgetting. These results suggest that explicitly modeling temporal prototype evolution is a practical and interpretable strategy for robust continual remote sensing segmentation.

ProtoFlow: Mitigating Forgetting in Class-Incremental Remote Sensing Segmentation via Low-Curvature Prototype Flow

Abstract

Remote sensing segmentation in real deployment is inherently continual: new semantic categories emerge, and acquisition conditions shift across seasons, cities, and sensors. Despite recent progress, many incremental approaches still treat training steps as isolated updates, which leaves representation drift and forgetting insufficiently controlled. We present ProtoFlow, a time-aware prototype dynamics framework that models class prototypes as trajectories and learns their evolution with an explicit temporal vector field. By jointly enforcing low-curvature motion and inter-class separation, ProtoFlow stabilizes prototype geometry throughout incremental learning. Experiments on standard class- and domain-incremental remote sensing benchmarks show consistent gains over strong baselines, including up to 1.5-2.0 points improvement in mIoUall, together with reduced forgetting. These results suggest that explicitly modeling temporal prototype evolution is a practical and interpretable strategy for robust continual remote sensing segmentation.

Paper Structure

This paper contains 47 sections, 6 theorems, 100 equations, 11 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Continual semantic segmentation.
  4. Prototype- and distribution-based continual learning.
  5. Remote-sensing segmentation and RS-specific continual learning.
  6. Method
  7. Problem Setting and Preliminaries
  8. Class-incremental semantic segmentation.
  9. ProtoFlow Field
  10. Prototype Flow Regularization: Low-Curvature Temporal Dynamics
  11. Segmentation, Distillation, and Joint Optimization
  12. Experimental Results
  13. Experimental Setup
  14. Main results
  15. Ablation and Analysis
  16. ...and 32 more sections

Key Result

Lemma 1

Under Assumption assump:gaussian, for each step $t$, the misclassification probability of class $c$ satisfies $\blacktriangleleft$$\blacktriangleleft$

Figures (11)

  • Figure 1: Existing RS CISS treats each step as an isolated snapshot and heuristically corrects drifting prototypes, whereas ProtoFlow models class prototypes as time-driven trajectories and learns a vector field that explicitly shapes their flow geometry to reduce forgetting.
  • Figure 2: Overall ProtoFlow framework. Non-stationary RS streams bring new classes over time. A segmentation network produces pixel features, which are aggregated by a prototype estimator and stored in a prototype bank. A time-aware ProtoFlow Field predicts how historical prototypes should move, and a prototype regularizer enforces flow consistency, low curvature and class separation. These prototype losses are combined with standard segmentation and distillation losses to jointly update the network and flow field, stabilizing old classes while learning new ones.
  • Figure 3: Per-class correlation between prototype trajectory curvature and forgetting.
  • Figure 4: Impact of time shuffling on LoveDA.
  • Figure 5: Prototype flow visualization on DeepGlobe. We project class prototypes into 2D and visualize their trajectories across incremental steps. (a) ProtoFlow: trajectories are smooth, monotone, and well separated. (b) w/o Curvature: individual trajectories exhibit stronger self-wrapping, oscillation, and backtracking. (c) w/o Separation: trajectories are increasingly crowded into a shared region, resulting in reduced margins and more cross-class intersections. (d) w/o Time: trajectories show stage-wise direction mismatch and stronger turning/correction patterns. Open circles and filled squares denote the first and last incremental steps, respectively.
  • ...and 6 more figures

Theorems & Definitions (13)

  • Definition 1
  • Lemma 1: Gaussian margin bound
  • Proof 1
  • Lemma 2: Lipschitz continuity of $g$
  • Proof 2
  • Lemma 3: Margin is Lipschitz in path length
  • Proof 3
  • Lemma 4
  • Proof 4
  • Theorem 1: Curvature control of forgetting
  • ...and 3 more