EarthSight: A Distributed Framework for Low-Latency Satellite Intelligence

TL;DR

EarthSight tackles the latency and scalability bottlenecks of satellite imagery analysis by redesigning onboard processing as a distributed ground–orbit decision problem. It introduces a multi-task onboard model with a shared backbone, a ground-station look-ahead scheduler, and an adaptive, utility-driven on-orbit filter ordering that prioritizes high-value imagery under strict power and bandwidth constraints. The approach combines global context with local adaptivity, using a greedy SBFE-inspired policy and a compact, pipelined execution model to reduce per-image compute time and tail latency, achieving a 1.9x speedup and cutting P90 end-to-end latency from 51 to 21 minutes in simulations and hardware-in-the-loop evaluations. The results demonstrate substantial improvements across multiple scenarios and confirm that ground-scheduled, multi-task constellation planning can scale to diverse, time-sensitive missions while maintaining energy efficiency and downlink efficiency. This framework lays the groundwork for scalable, low-latency satellite intelligence capable of supporting disaster response, security, and urban monitoring in real-world deployments.

Abstract

Low-latency delivery of satellite imagery is essential for time-critical applications such as disaster response, intelligence, and infrastructure monitoring. However, traditional pipelines rely on downlinking all captured images before analysis, introducing delays of hours to days due to restricted communication bandwidth. To address these bottlenecks, emerging systems perform onboard machine learning to prioritize which images to transmit. However, these solutions typically treat each satellite as an isolated compute node, limiting scalability and efficiency. Redundant inference across satellites and tasks further strains onboard power and compute costs, constraining mission scope and responsiveness. We present EarthSight, a distributed runtime framework that redefines satellite image intelligence as a distributed decision problem between orbit and ground. EarthSight introduces three core innovations: (1) multi-task inference on satellites using shared backbones to amortize computation across multiple vision tasks; (2) a ground-station query scheduler that aggregates user requests, predicts priorities, and assigns compute budgets to incoming imagery; and (3) dynamic filter ordering, which integrates model selectivity, accuracy, and execution cost to reject low-value images early and conserve resources. EarthSight leverages global context from ground stations and resource-aware adaptive decisions in orbit to enable constellations to perform scalable, low-latency image analysis within strict downlink bandwidth and onboard power budgets. Evaluations using a prior established satellite simulator show that EarthSight reduces average compute time per image by 1.9x and lowers 90th percentile end-to-end latency from first contact to delivery from 51 to 21 minutes compared to the state-of-the-art baseline.

Figures (11)

• Figure 1: LEO Earth observation systems perform on-board inference for image analysis. Ground stations send tasks generated from user queries to satellites. The satellites prioritize transmission of images identified as latency-sensitive via ML inference.
• Figure 2: Overview of the EarthSight system. EarthSight integrates onboard multi-task models with predictive scheduling to enable query-driven, low-latency analysis of satellite images.
• Figure 3: EarthSight’s multi-task model architecture. A shared backbone processes the input image to produce a latent representation, which is used by lightweight, task-specific heads to perform heterogeneous tasks such as classification.
• Figure 4: Timing diagrams illustrating filter execution strategies on a CPU-xPU system: (a) Serval's (baseline) sequential execution serval, (b) Pipelined baseline with overlapped loading, (c) EarthSight with predictive filter selection and loading, and (d) EarthSight with a filter misprediction. The phases represent Filter Selection (runtime scheduling), Filter Loading (CPU model preparation), Filter Execution (xPU inference), and Communication Overhead (CPU-xPU coordination).
• Figure 5: Comparison of model size and performance across single-task and multi-task configurations.