Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Embedding-Only Uplink for Onboard Retrieval Under Shift in Remote Sensing

Sangcheol Sim

Abstract

Downlink bottlenecks motivate onboard systems that prioritize hazards without transmitting raw pixels. We study a strict setting where a ground station uplinks only compact embeddings plus metadata, and an onboard system performs vector search to triage new captures. We ask whether this embedding-only pipeline remains useful under explicit remote-sensing shift: cross-time (pre/post-event), cross-event/location (different disasters), cross-site cloud (15 geographic sites), and cross-city AOI holdout (buildings). Using OlmoEarth embeddings on a scaled public multi-task benchmark (27 Sentinel-2 L2A scenes, 15 cloud sites, 5 SpaceNet-2 AOIs; 10 seeds), we find that all effective methods rely on the same uplinked embeddings, but the optimal decision head is task-dependent: kNN retrieval is significantly superior for cloud classification (0.92 vs. centroid 0.91; p<0.01, Wilcoxon), while class centroids dominate temporal change detection (0.85 vs. retrieval 0.48; p<0.01). These results show that embedding-only uplink is the key enabler--once embeddings are onboard, the system can select the best head per task at no additional uplink cost, with all telemetry under 1 KB per query.

Embedding-Only Uplink for Onboard Retrieval Under Shift in Remote Sensing

Abstract

Downlink bottlenecks motivate onboard systems that prioritize hazards without transmitting raw pixels. We study a strict setting where a ground station uplinks only compact embeddings plus metadata, and an onboard system performs vector search to triage new captures. We ask whether this embedding-only pipeline remains useful under explicit remote-sensing shift: cross-time (pre/post-event), cross-event/location (different disasters), cross-site cloud (15 geographic sites), and cross-city AOI holdout (buildings). Using OlmoEarth embeddings on a scaled public multi-task benchmark (27 Sentinel-2 L2A scenes, 15 cloud sites, 5 SpaceNet-2 AOIs; 10 seeds), we find that all effective methods rely on the same uplinked embeddings, but the optimal decision head is task-dependent: kNN retrieval is significantly superior for cloud classification (0.92 vs. centroid 0.91; p<0.01, Wilcoxon), while class centroids dominate temporal change detection (0.85 vs. retrieval 0.48; p<0.01). These results show that embedding-only uplink is the key enabler--once embeddings are onboard, the system can select the best head per task at no additional uplink cost, with all telemetry under 1 KB per query.

Paper Structure

This paper contains 10 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. Embedding-only uplink and onboard indexing
  4. Multi-modal embeddings and decision head
  5. Compact, auditable telemetry
  6. Benchmark and metrics
  7. Multi-task public benchmark
  8. Metrics
  9. Results
  10. Conclusion

Figures (3)

  • Figure 1: Embedding-only uplink pipeline. The ground station computes hint embeddings and uplinks (embedding, metadata) tuples---no imagery. The onboard system indexes hints in a vector database and retrieves top-$k$ matches for each new capture via cosine similarity, emitting compact JSON telemetry (${\sim}$700 B at $k{=}5$) as the downlink product.
  • Figure 2: Qualitative retrieval example (hazard task). A Derna flood query retrieves same-event crops (sim$>$0.96) before a cross-event match (Maui wildfire, sim$=$0.91), demonstrating embedding-level hazard grouping.
  • Figure 3: $k$-sweep: task metric vs. telemetry bytes ($k\in\{1,5,10\}$). Dashed horizontal lines show $k$-independent baselines (centroid, linear probe). Buildings favors small $k$; change improves with larger $k$. Error bars: $\pm$1 std over 10 seeds.