Remote Sensing Object Counting with Online Knowledge Learning

TL;DR

This work tackles remote sensing object counting under resource constraints by introducing OnKL Net, an end-to-end online knowledge learning framework that fuses a pre-trained teacher branch with a channel-reduced student branch through a shared shallow module. A novel relation-in-relation distillation (RiRD) transfers not only feature representations but also the evolution of intra-feature relationships across layers, enabling a lightweight student to approach teacher-level performance. Extensive experiments on RSOC and STAR show that OnKL Net achieves competitive accuracy with far fewer parameters and lower training time than traditional two-stage distillation methods, while providing explicit object locations. The approach holds practical promise for deployment on drones and embedded systems where computational budgets are strict, and it offers a new direction for distillation through inter-layer relationship modeling.

Abstract

Efficient models for remote sensing object counting are urgently required for applications in scenarios with limited computing resources, such as drones or embedded systems. A straightforward yet powerful technique to achieve this is knowledge distillation, which steers the learning of student networks by leveraging the experience of already-trained teacher networks. However, it faces a pair of challenges: Firstly, due to its two-stage training nature, a longer training period is essential, especially as the training samples increase. Secondly, despite the proficiency of teacher networks in transmitting assimilated knowledge, they tend to overlook the latent insights gained during their learning process. To address these challenges, we introduce an online distillation learning method for remote sensing object counting. It builds an end-to-end training framework that seamlessly integrates two distinct networks into a unified one. It comprises a shared shallow module, a teacher branch, and a student branch. The shared module serving as the foundation for both branches is dedicated to learning some primitive information. The teacher branch utilizes prior knowledge to reduce the difficulty of learning and guides the student branch in online learning. In parallel, the student branch achieves parameter reduction and rapid inference capabilities by means of channel reduction. This design empowers the student branch not only to receive privileged insights from the teacher branch but also to tap into the latent reservoir of knowledge held by the teacher branch during the learning process. Moreover, we propose a relation-in-relation distillation method that allows the student branch to effectively comprehend the evolution of the relationship of intra-layer teacher features among different inter-layer features. Extensive experiments demonstrate the effectiveness of our method.

Figures (6)

• Figure 1: Comparison of distillation pipeline (traditional method vs. our method). (a) The traditional method involves training the teacher network first, followed by training the student network; (b) Our method employs an online distillation method to train the teacher and student branches jointly.
• Figure 2: An overview of the proposed online distillation network. It consists of the shared shallow module, teacher branch and student branch. Meanwhile, feature distillation and relation-in-relation distillation are introduced to guide knowledge transfer from the teacher branch to the student branch.
• Figure 3: Overview of relation-in-relation feature transformation. The inputs come from two different feature maps, $t_i (s_i^\prime)$ and $t_j (s_j^\prime)$, from different groups in the teacher (or student) branch, where $1 \le i < j \le M$.
• Figure 4: Visualization of the inter-layer relationship matrix. The relationship matrices of the teacher and student branches are represented in the first and second rows, respectively. ${t_i} \rightarrow {t_j} ({s_i} \rightarrow {s_j})$ denotes the relation matrix generated by ${t_i} ({s_i})$ and ${t_j} ({s_j})$. To clearly highlight between strong and weak relevant relationships, we apply the transformation $-1/log(x)$ to the non-zero values of the relationship matrix.
• Figure 5: Comparison of predicted results on Building and Small-vehicle. The first three rows show the images sampled from Building, while the remaining rows feature images of Small-vehicle.