GLACIA: Instance-Aware Positional Reasoning for Glacial Lake Segmentation via Multimodal Large Language Model

TL;DR

GLACIA introduces a first-of-its-kind framework that merges multispectral segmentation with multimodal language-driven reasoning to produce both precise glacial lake masks and interpretable positional descriptions. By coupling a Prithvi-Res multispectral encoder with a multimodal LLM and a Prompt Mask Decoder, it delivers instance-aware, spatially grounded segmentation while generating natural language explanations of lake counts and locations. The GLake-Pos dataset provides the necessary reasoning data for training, enabling robust region-level grounding beyond pixel accuracy. Experimental results show GLACIA surpasses CNNs, ViTs, and existing reasoning-based methods in both segmentation performance and contextual reasoning metrics, highlighting its potential for improved disaster preparedness and policy-making in evolving glacial environments.

Abstract

Glacial lake monitoring bears great significance in mitigating the anticipated risk of Glacial Lake Outburst Floods. However, existing segmentation methods based on convolutional neural networks (CNNs) and Vision Transformers (ViTs), remain constrained to pixel-level predictions, lacking high-level global scene semantics and human-interpretable reasoning. To address this, we introduce GLACIA (\textbf{G}lacial \textbf{LA}ke segmentation with \textbf{C}ontextual \textbf{I}nstance \textbf{A}wareness), the first framework that integrates large language models with segmentation capabilities to produce both accurate segmentation masks and corresponding spatial reasoning outputs. We construct the Glacial Lake Position Reasoning (GLake-Pos) dataset pipeline, which provides diverse, spatially grounded question-answer pairs designed to overcome the lack of instance-aware positional reasoning data in remote sensing. Comparative evaluation demonstrate that GLACIA (mIoU: 87.30) surpasses state-of-the-art method based on CNNs (mIoU: 78.55 - 79.01), ViTs (mIoU: 69.27 - 81.75), Geo-foundation models (mIoU: 76.37 - 87.10), and reasoning based segmentation methods (mIoU: 60.12 - 75.66). Our approach enables intuitive disaster preparedness and informed policy-making in the context of rapidly changing glacial environments by facilitating natural language interaction, thereby supporting more efficient and interpretable decision-making. The code is released on https://github.com/lalitmaurya47/GLACIA

Figures (5)

• Figure 1: Conceptual shift from traditional segmentation (a) and VQA-based reasoning (b) to our reasoning-driven paradigm (c), which unifies accurate instance-specific masks with interpretable positional reasoning.
• Figure 2: Overview of the proposed architecture for glacial lake segmentation. (a) Prithvi-Res encoder fuses multispectral local and global context, while (b) multimodal LLM generates segmentation-specific tokens from RGB imagery and text. (c) Prompt Mask Decoder aligns these tokens with multispectral features to produce precise spatial masks.
• Figure 3: Qualitative comparison of glacial lake segmentation. Red boxes indicate false positives, blue boxes indicate false negatives. Our reasoning-enhanced model accurately captures small and irregular lakes, reducing both errors compared to baseline methods.
• Figure 4: Examples of reasoning outputs and corresponding segmentation masks from test samples used in evaluation.
• Figure A.5: The position of Glacial Lake form the center