Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Parameter-Efficient Mixture-of-Experts Framework for Cross-Modal Geo-Localization

LinFeng Li, Jian Zhao, Zepeng Yang, Yuhang Song, Bojun Lin, Tianle Zhang, Yuchen Yuan, Chi Zhang, Xuelong Li

TL;DR

This work tackles cross-modal geo-localization across heterogeneous platforms (satellite, drone, ground) by addressing appearance and textual-domain gaps with a Parameter-Efficient Mixture-of-Experts (PE-MoE) framework. It combines domain-aligned preprocessing (platform-based data stratification, LLM-based caption refinement, satellite-specific augmentations) with frozen backbones (BGE-M3 for text, EVA-CLIP for images) and lightweight, platform-specific expert heads, all orchestrated by a dynamic gating network. A two-stage training regime (base contrastive pretraining followed by hard-negative mining) and a gated inference that fuses per-expert similarities yield state-of-the-art results on RoboSense 2025 Track 4, validating robustness under severe viewpoint and modality shifts. The approach demonstrates the practical value of dividing a complex cross-modal task into specialized, parameter-efficient components, with implications for scalable, real-world geo-localization systems in autonomous navigation.

Abstract

We present a winning solution to RoboSense 2025 Track 4: Cross-Modal Drone Navigation. The task retrieves the most relevant geo-referenced image from a large multi-platform corpus (satellite/drone/ground) given a natural-language query. Two obstacles are severe inter-platform heterogeneity and a domain gap between generic training descriptions and platform-specific test queries. We mitigate these with a domain-aligned preprocessing pipeline and a Mixture-of-Experts (MoE) framework: (i) platform-wise partitioning, satellite augmentation, and removal of orientation words; (ii) an LLM-based caption refinement pipeline to align textual semantics with the distinct visual characteristics of each platform. Using BGE-M3 (text) and EVA-CLIP (image), we train three platform experts using a progressive two-stage, hard-negative mining strategy to enhance discriminative power, and fuse their scores at inference. The system tops the official leaderboard, demonstrating robust cross-modal geo-localization under heterogeneous viewpoints.

A Parameter-Efficient Mixture-of-Experts Framework for Cross-Modal Geo-Localization

TL;DR

This work tackles cross-modal geo-localization across heterogeneous platforms (satellite, drone, ground) by addressing appearance and textual-domain gaps with a Parameter-Efficient Mixture-of-Experts (PE-MoE) framework. It combines domain-aligned preprocessing (platform-based data stratification, LLM-based caption refinement, satellite-specific augmentations) with frozen backbones (BGE-M3 for text, EVA-CLIP for images) and lightweight, platform-specific expert heads, all orchestrated by a dynamic gating network. A two-stage training regime (base contrastive pretraining followed by hard-negative mining) and a gated inference that fuses per-expert similarities yield state-of-the-art results on RoboSense 2025 Track 4, validating robustness under severe viewpoint and modality shifts. The approach demonstrates the practical value of dividing a complex cross-modal task into specialized, parameter-efficient components, with implications for scalable, real-world geo-localization systems in autonomous navigation.

Abstract

We present a winning solution to RoboSense 2025 Track 4: Cross-Modal Drone Navigation. The task retrieves the most relevant geo-referenced image from a large multi-platform corpus (satellite/drone/ground) given a natural-language query. Two obstacles are severe inter-platform heterogeneity and a domain gap between generic training descriptions and platform-specific test queries. We mitigate these with a domain-aligned preprocessing pipeline and a Mixture-of-Experts (MoE) framework: (i) platform-wise partitioning, satellite augmentation, and removal of orientation words; (ii) an LLM-based caption refinement pipeline to align textual semantics with the distinct visual characteristics of each platform. Using BGE-M3 (text) and EVA-CLIP (image), we train three platform experts using a progressive two-stage, hard-negative mining strategy to enhance discriminative power, and fuse their scores at inference. The system tops the official leaderboard, demonstrating robust cross-modal geo-localization under heterogeneous viewpoints.
Paper Structure (26 sections, 1 equation, 2 figures, 2 tables)

This paper contains 26 sections, 1 equation, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Cross-Modal Image-Text Retrieval
  4. Visual Geo-Localization
  5. Mixture-of-Experts and Model Ensembles
  6. Method
  7. Data Preprocessing and Alignment
  8. Platform-based Data Stratification
  9. Textual Domain Alignment
  10. Augmentation and Sanitization for Satellite Imagery
  11. Parameter-Efficient MoE Framework
  12. Shared Encoder Backbones
  13. Differentiated Expert Heads
  14. Dynamic Gating Network
  15. Training and Inference
  16. ...and 11 more sections

Figures (2)

  • Figure 1: The overall architecture of our Parameter-Efficient Mixture-of-Experts (PE-MoE) framework. A shared backbone extracts general features, which are processed by a dynamic gating network and specialized expert heads to produce the final retrieval score.
  • Figure 2: The two-stage training pipeline. Stage 1 builds general alignment using positive pairs, while Stage 2 uses mined hard negatives to refine the model's discriminative ability.