Rebellious Student: A Complementary Learning Framework for Background Feature Enhancement in Hyperspectral Anomaly Detection

TL;DR

The paper tackles hyperspectral anomaly detection by enhancing background priors through complementary spectral-spatial features. It introduces a two-stage Rebellious Student framework: a spectral Feature Enhancement Network (Spe-FEN) trained via reverse distillation and Spectral Feature Alignment Mechanism, and a spatial Feature Enhancement Network (Spa-FEN) trained as a rebellious student using decorrelation and reconstruction losses to learn spatial cues not captured by the spectral teacher. Through two fusion strategies and RX-based anomaly scoring, the method achieves state-of-the-art performance on HAD100 with only modest computational overhead and demonstrates improved Gaussian-like distribution characteristics that generalize across scenes. The work highlights the potential of purposely diverging representations to achieve stronger background modeling and broadens the applicability of background-feature enhancement in remote sensing tasks.

Abstract

A recent class of hyperspectral anomaly detection methods that can be trained once on background datasets and then universally deployed -- without per-scene retraining or parameter tuning -- has demonstrated remarkable efficiency and robustness. Building upon this paradigm, we focus on the integration of spectral and spatial cues and introduce a novel "Rebellious Student" framework for complementary feature learning. Unlike conventional teacher-student paradigms driven by imitation, our method intentionally trains the spatial branch to diverge from the spectral teacher, thereby learning complementary spatial patterns that the teacher fails to capture. A two-stage learning strategy is adopted: (1) a spectral enhancement network is first trained via reverse distillation to obtain robust background spectral representations; and (2) a spatial network -- the rebellious student -- is subsequently optimized using decorrelation losses that enforce feature orthogonality while maintaining reconstruction fidelity to avoid irrelevant noise. Once trained, the framework enhances both spectral and spatial background features, enabling parameter-free and training-free anomaly detection when paired with conventional detectors. Experiments on the HAD100 benchmark show substantial improvements over several established baselines with modest computational overhead, confirming the effectiveness of the proposed complementary learning paradigm. Our code is publicly available at https://github.com/xjpp2016/FERS.

Figures (9)

• Figure 1: Schematic illustration of the proposed “rebellious student” framework. Unlike conventional student-teacher paradigms, our approach treats one branch as a rebellious student, which is encouraged to diverge from the teacher's output and learn complementary information that the teacher overlooks.
• Figure 2: Illustration of the implementation of the rebellious student framework in our method. In Step 1, the spectral Feature Enhancement Network (FEN) is obtained via FERD and used as the teacher network in Step 2. In Step 2, the spatial FEN is trained as the “rebellious student” to capture complementary spatial features.
• Figure 3: An illustration of the encoder-decoder with reverse distillation and the Spectral Feature Alignment Mechanism (SFAM) in Stage 1 (Spectral Branch). The reverse distillation differs from a standard encoder-decoder as most of the encoder is frozen, analogous to the teacher in knowledge distillation. However, it diverges from standard distillation by having a reversed data flow in the student (decoder). The SFAM, primarily realized through the Zero Centering Loss and its auxiliary structures, aims to softly align the most spectrally-relevant features from the encoder and decoder in the spectral space.
• Figure 4: The pruned Spectral Feature Enhancement Network (Spe-FEN), which retains only the components most relevant to spectral feature extraction—the shallow layers of the encoder and the Feature Projection Layer (FPL) from the SFAM. Its architecture predominantly comprises 1×1 convolutions, with only a few 3×3 convolutions, ensuring its focus on modeling spectral characteristics.
• Figure 5: Architecture and training process of the Spatial Branch (Stage 2). During this stage, the pre-trained Spectral Feature Enhancement Network (Spe-FEN) serves as a fixed teacher, while the Spatial Feature Enhancement Network (Spa-FEN) acts as a "rebellious student" to learn spatial features complementary to the teacher's spectral expertise. The Spa-FEN comprises: a spectral compression module to reduce interference, a ResNet-34 for high-level spatial feature extraction, and a LightSPP module for multi-scale context aggregation. A decoder composed of stacked channel attention and residual convolutional layers then fuses the features. The training objective combines a decorrelation loss $\mathcal{L}_1$ to encourage feature independence between the teacher and student, and a reconstruction loss $\mathcal{L}_2$ to ensure the student outputs meaningful features for HSI reconstruction.