VICoT-Agent: A Vision-Interleaved Chain-of-Thought Framework for Interpretable Multimodal Reasoning and Scalable Remote Sensing Analysis

TL;DR

VICoT introduces a vision-interleaved, multi-round reasoning framework that unifies reasoning and tool invocation via a stack-based memory and an MCP XML tool protocol. It enables explicit, interpretable interaction between large language models and a modular set of visual tools for remote sensing analysis, including region-aware processing for ultra-high-resolution imagery. A Reasoning Stack Distillation pipeline transfers complex multimodal reasoning to lightweight edge models, and Region-Aware Captioned Prompting enhances regional understanding within large scenes. Across five RS datasets, VICoT demonstrates superior trajectory quality, robust report quality, and improved efficiency over state-of-the-art baselines, with distilled variants enabling practical edge deployment.

Abstract

The current remote sensing image analysis task is increasingly evolving from traditional object recognition to complex intelligence reasoning, which places higher requirements on the model's reasoning ability and the flexibility of tool invocation. To this end, we propose a new multimodal agent framework, Vision-Interleaved Chain-of-Thought Framework (VICoT), which implements explicit multi-round reasoning by dynamically incorporating visual tools into the chain of thought. Through a stack-based reasoning structure and a modular MCP-compatible tool suite, VICoT enables LLMs to efficiently perform multi-round, interleaved vision-language reasoning tasks with strong generalization and flexibility.We also propose the Reasoning Stack distillation method to migrate complex Agent behaviors to small, lightweight models, which ensures the reasoning capability while significantly reducing complexity. Experiments on multiple remote sensing benchmarks demonstrate that VICoT significantly outperforms existing SOTA frameworks in reasoning transparency, execution efficiency, and generation quality.

Figures (5)

• Figure 1: Example of multi-round reasoning and tool invocation by the VICoT Agent in a remote sensing image analysis task. The agent begins by describing the overall scene and identifying relevant objects using an open-vocabulary detector (Steps 1–2). It then applies cropping (Step 3) and super-resolution (Step 4) to refine visual details, followed by augmented aircraft type recognition (Step 5) and hull number identification (Step 6). Based on extracted keywords, it invokes a web search tool to retrieve supporting background information (Step 7), final in a SOAP-formatted intelligence report. Each reasoning step is grounded in a specific visual region and associated tool output, demonstrating the interpretability and transparency.
• Figure 2: Illustration of the interaction between the Reasoning Stack and the MCP Tool Set in the VICoT framework: The agent operates through a multi-round reasoning process, where each step recorded in the Reasoning Stack(left panel) includes the decision made by the LLM-based Think Module and the corresponding tool invocation. MCP Tool Set(right panel) includes various vision and text tools such as object detection (OD), cropping (Crop), super-resolution (SR), denoising, binarization (Bin), cloud/rain removal (CR/RR), motion deblurring (MD), web search (WS), and RAG-based retrieval. Each tool is invoked via standardized XML under MCP protocol. This design enables modular, interpretable decision-making and flexible tool usage throughout the multi-round reasoning process.
• Figure 3: The RS tool-set of the VICoT Agent: The figure shows an output example of the OD tool (top left), as well as the enhancement effects of the Crop, Binarization, and Super-Resolution tools on fine-grained image details.
• Figure 4: Illustration of Reasoning Stack distillation. The teacher model (left) generates a complete Reasoning Stack, which is used to supervise the student model (right).
• Figure 5: Isolation Agent Workflow