Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Heterogeneous Mixture of Experts for Remote Sensing Image Super-Resolution

Bowen Chen, Keyan Chen, Mohan Yang, Zhengxia Zou, Zhenwei Shi

TL;DR

This work tackles the challenge of reconstructing high-resolution remote-sensing images by addressing the heterogeneous nature of ground objects. It introduces a Multi-level Feature Guided HMoE (MFG-HMoE) that employs heterogeneous expert groups, a multi-level feature aggregation (MFA) module to guide routing, and a dual-routing mechanism to assign each pixel to an optimal expert. The approach yields state-of-the-art SR performance on UCMerced and AID, with ablation studies confirming the effectiveness of the heterogeneous experts, MFA, and two-stage routing. The proposed framework enhances per-pixel reconstruction fidelity for diverse geospatial content, offering strong practical impact for downstream remote-sensing tasks.

Abstract

Remote sensing image super-resolution (SR) aims to reconstruct high-resolution remote sensing images from low-resolution inputs, thereby addressing limitations imposed by sensors and imaging conditions. However, the inherent characteristics of remote sensing images, including diverse ground object types and complex details, pose significant challenges to achieving high-quality reconstruction. Existing methods typically employ a uniform structure to process various types of ground objects without distinction, making it difficult to adapt to the complex characteristics of remote sensing images. To address this issue, we introduce a Mixture of Experts (MoE) model and design a set of heterogeneous experts. These experts are organized into multiple expert groups, where experts within each group are homogeneous while being heterogeneous across groups. This design ensures that specialized activation parameters can be employed to handle the diverse and intricate details of ground objects effectively. To better accommodate the heterogeneous experts, we propose a multi-level feature aggregation strategy to guide the routing process. Additionally, we develop a dual-routing mechanism to adaptively select the optimal expert for each pixel. Experiments conducted on the UCMerced and AID datasets demonstrate that our proposed method achieves superior SR reconstruction accuracy compared to state-of-the-art methods. The code will be available at https://github.com/Mr-Bamboo/MFG-HMoE.

Heterogeneous Mixture of Experts for Remote Sensing Image Super-Resolution

TL;DR

This work tackles the challenge of reconstructing high-resolution remote-sensing images by addressing the heterogeneous nature of ground objects. It introduces a Multi-level Feature Guided HMoE (MFG-HMoE) that employs heterogeneous expert groups, a multi-level feature aggregation (MFA) module to guide routing, and a dual-routing mechanism to assign each pixel to an optimal expert. The approach yields state-of-the-art SR performance on UCMerced and AID, with ablation studies confirming the effectiveness of the heterogeneous experts, MFA, and two-stage routing. The proposed framework enhances per-pixel reconstruction fidelity for diverse geospatial content, offering strong practical impact for downstream remote-sensing tasks.

Abstract

Remote sensing image super-resolution (SR) aims to reconstruct high-resolution remote sensing images from low-resolution inputs, thereby addressing limitations imposed by sensors and imaging conditions. However, the inherent characteristics of remote sensing images, including diverse ground object types and complex details, pose significant challenges to achieving high-quality reconstruction. Existing methods typically employ a uniform structure to process various types of ground objects without distinction, making it difficult to adapt to the complex characteristics of remote sensing images. To address this issue, we introduce a Mixture of Experts (MoE) model and design a set of heterogeneous experts. These experts are organized into multiple expert groups, where experts within each group are homogeneous while being heterogeneous across groups. This design ensures that specialized activation parameters can be employed to handle the diverse and intricate details of ground objects effectively. To better accommodate the heterogeneous experts, we propose a multi-level feature aggregation strategy to guide the routing process. Additionally, we develop a dual-routing mechanism to adaptively select the optimal expert for each pixel. Experiments conducted on the UCMerced and AID datasets demonstrate that our proposed method achieves superior SR reconstruction accuracy compared to state-of-the-art methods. The code will be available at https://github.com/Mr-Bamboo/MFG-HMoE.

Paper Structure

This paper contains 12 sections, 7 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Overview
  4. Multi-level Feature Aggregation
  5. Dual-Routing Heterogeneous Mixture of Experts
  6. Heterogeneous Experts
  7. Dual-Routing Mechanism
  8. Experiments
  9. Datasets and Experimental Setup
  10. Comparison with Other Methods
  11. Ablation Studies
  12. Conclusion

Figures (3)

  • Figure 1: The flowchart of the proposed MFG-HMoE. The feature extraction network is constructed by stacking RHAGs chen2023activating. The MFA module aggregates the multi-level features from the feature extraction network and feeds them into the two routers. DR-HMoE routes each feature pixel output from the feature extraction network to the optimal expert for processing.
  • Figure 2: Comparison of results from different methods in $\times$4 SR on the UCMerced dataset with the HR ground truth.
  • Figure 3: Visualization of the selected experts in $\times$4 SR on the UCMerced dataset.