SmartScan: An AI-based Interactive Framework for Automated Region Extraction from Satellite Images

TL;DR

The paper tackles the scalability challenge of planning fixed methane sensor networks by automating region extraction from ultra-high-resolution satellite imagery. It introduces SmartScan, an AI-based interactive framework that uses the Segment Anything Model (SAM) to extract subspaces of interest, with two operating modes: Data Curation (interactive prompting to generate high-quality SAM outputs) and Autonomous (a lightweight network trained on curated prompts to replace manual prompting). The pipeline includes satellite image extraction, prompt generation (manual and autonomous), post-processing to produce tight convex subspaces, and an interactive quality-check tool for defining facility-specific constraints, all in a domain-agnostic, high-throughput design. The approach offers improved scalability and efficiency for downstream sensor-placement and subspace-based source-inversion, with practical impact for rapid evaluation of multiple facilities using ultra-high-resolution imagery.

Abstract

The deployment of a continuous methane monitoring system requires determining the optimal number and placement of fixed sensors. However, planning is labor-intensive, requiring extensive site setup and iteration to meet client restrictions. This challenge is amplified when evaluating multiple sites, limiting scalability. To address this, we introduce SmartScan, an AI framework that automates data extraction for optimal sensor placement. SmartScan identifies subspaces of interest from satellite images using an interactive tool to create facility-specific constraint sets efficiently. SmartScan leverages the Segment Anything Model (SAM), a prompt-based transformer for zero-shot segmentation, enabling subspace extraction without explicit training. It operates in two modes: (1) Data Curation Mode, where satellite images are processed to extract high-quality subspaces using an interactive prompting system for SAM, and (2) Autonomous Mode, where user-curated prompts train a deep learning network to replace manual prompting, fully automating subspace extraction. The interactive tool also serves for quality control, allowing users to refine AI-generated outputs and generate additional constraint sets as needed. With its AI-driven prompting mechanism, SmartScan delivers high-throughput, high-quality subspace extraction with minimal human intervention, enhancing scalability and efficiency. Notably, its adaptable design makes it suitable for extracting regions of interest from ultra-high-resolution satellite imagery across various domains.

Figures (12)

• Figure 1: Example of a continuous methane leak monitoring system. An optimal number of sensors have been deployed at the facility according to the generated optimal placement design. Further, targeted source-inversion is used to determine the subspace inside which a leak occurs.
• Figure 2: Continuous real-time monitoring system workflow. Client-provided GPS coordinates are given to module A, where facility-specific constraint sets are generated. These are given to module B to generate an optimal sensor placement design. The proposed sensor placement design is converted to GPS coordinates, and deployed at the facility. Finally, targeted source-inversion happens in module C for identifying the subspace in which a leak occurs with continuous monitoring.
• Figure 3: Satellite images of different client facilities. It can be observed that each facility is visually dissimilar from one-another, which makes the process of manually defining subspaces labor intensive.
• Figure 4: Segment Anything Mode (SAM) overview. A heavy-weight image encoder outputs an image embedding that can be efficiently queried by a variety of input prompts to produce object masks at amortized real-time speed.
• Figure 5: SmartScan Back-End Pipeline. SmartScan is an end-to-end, interactive tool for semi-automated salient region extraction from satellite images. Our tool provides a full stack solution for accurately extracting convex polygons around salient regions of interest, given the GPS coordinates of the site. At its core, SmartScan uses the Segment Anything Model (SAM) for prompt-based segmentation of regions of interest from a satellite image. The generated convex polygon set is used for optimal sensor placement design.