Insight Any Instance: Promptable Instance Segmentation for Remote Sensing Images
Xuexue Li
TL;DR
RSI instance segmentation struggles with extreme foreground-background imbalance and tiny instances, compounded by downsampling in conventional deep-feature pipelines. The authors introduce a promptable instance segmentation framework comprising a Local Prompt Module (LPM) that extracts instance-focused texture cues from un-downsampled regions and a Global-to-Local Prompt Module (GPM) that injects global context into local instance tokens, complemented by a Proposals' Area Loss (PAreaLoss) to refine proposals. The approach supports interactive box prompts for promptable segmentation and delivers fast inference (~40 ms) while achieving competitive or superior results across four RSI datasets (optical and SAR). This work demonstrates that instance-specific prompts can compensate for downsampling drawbacks, offering robust RSI segmentation with modest computational overhead and practical deployment potential.
Abstract
Instance segmentation of remote sensing images (RSIs) is an essential task for a wide range of applications such as land planning and intelligent transport. Instance segmentation of RSIs is constantly plagued by the unbalanced ratio of foreground and background and limited instance size. And most of the instance segmentation models are based on deep feature learning and contain operations such as multiple downsampling, which is harmful to instance segmentation of RSIs, and thus the performance is still limited. Inspired by the recent superior performance of prompt learning in visual tasks, we propose a new prompt paradigm to address the above issues. Based on the existing instance segmentation model, firstly, a local prompt module is designed to mine local prompt information from original local tokens for specific instances; secondly, a global-to-local prompt module is designed to model the contextual information from the global tokens to the local tokens where the instances are located for specific instances. Finally, a proposal's area loss function is designed to add a decoupling dimension for proposals on the scale to better exploit the potential of the above two prompt modules. It is worth mentioning that our proposed approach can extend the instance segmentation model to a promptable instance segmentation model, i.e., to segment the instances with the specific boxes prompt. The time consumption for each promptable instance segmentation process is only 40 ms. The paper evaluates the effectiveness of our proposed approach based on several existing models in four instance segmentation datasets of RSIs, and thorough experiments prove that our proposed approach is effective for addressing the above issues and is a competitive model for instance segmentation of RSIs.