PROGRESSLM: Towards Progress Reasoning in Vision-Language Models

TL;DR

Progress-LM investigates estimating how much of a task has been completed from a single observation, framing progress estimation as long-horizon reasoning. It introduces Progress-Bench, a benchmark that probes perception, temporal reasoning, and uncertainty via controlled variations in demonstration modality, viewpoint, and answerability, and presents a human-inspired two-stage reasoning paradigm (episodic retrieval and mental simulation). The study shows current VLMs struggle with progress estimation, especially under modality shifts and unanswerable cases, and finds training-free prompting yields limited gains while a training-based ProgressLM-3B yields consistent improvements. The results highlight the value of explicit, coupled reasoning for robust progress estimation and point toward scalable improvements through supervised and reinforcement-learning training on purpose-built datasets. Overall, Progress-Bench and ProgressLM provide a framework for advancing dynamic, long-horizon reasoning in vision-language systems with implications for reliability and interpretability in real-world tasks.

Abstract

Estimating task progress requires reasoning over long-horizon dynamics rather than recognizing static visual content. While modern Vision-Language Models (VLMs) excel at describing what is visible, it remains unclear whether they can infer how far a task has progressed from partial observations. To this end, we introduce Progress-Bench, a benchmark for systematically evaluating progress reasoning in VLMs. Beyond benchmarking, we further explore a human-inspired two-stage progress reasoning paradigm through both training-free prompting and training-based approach based on curated dataset ProgressLM-45K. Experiments on 14 VLMs show that most models are not yet ready for task progress estimation, exhibiting sensitivity to demonstration modality and viewpoint changes, as well as poor handling of unanswerable cases. While training-free prompting that enforces structured progress reasoning yields limited and model-dependent gains, the training-based ProgressLM-3B achieves consistent improvements even at a small model scale, despite being trained on a task set fully disjoint from the evaluation tasks. Further analyses reveal characteristic error patterns and clarify when and why progress reasoning succeeds or fails.

Figures (20)

• Figure 1: Given a task demonstration and a single observation, the goal is to estimate how much of the task has already been completed. Direct prediction can often judge whether the task is unfinished, but struggles to assign a well-calibrated progress score. Progress reasoning instead follows a coarse-to-fine process: it first performs episodic retrieval to coarsely locate the observation along the demonstrated task, then applies mental simulation to imagine the transition from the retrieved anchor to the current observation, enabling a fine-grained estimate of completed progress, which enables accurate and interpretable progress estimation.
• Figure 2: Overview of Progress-Bench construction. (a) Demonstration setup: tasks are presented as either vision-based demonstrations with key frames or text-based ones with step-wise actions, each annotated with progress scores. (b) Observation sampling: observations are sampled from intermediate or boundary positions between demonstration steps, with progress labels assigned by interpolation; vision-based settings further include same-view and cross-view demonstration–observation correspondence. (c) Answerability augmentation: unanswerable cases are created by introducing semantic mismatches between demonstrations and observations.
• Figure 3: Data statistics of Progress-Bench and ProgressLM-45K (25K for SFT while 20K for RL). Traj and Samp denote the numbers of task trajectories and sampled observations to be estimated, respectively. The upper-right panel shows the four distinct robotic embodiments included, while the lower-right panel visualizes the diversity of objects involved in task interactions.
• Figure 4: Unanswerable Detection Accuracy (UDA) across models under two settings.
• Figure 5: Distribution of predicted progress scores. Some models exhibit collapsed or clustered distributions at extreme or discrete values, indicating reliance on heuristic anchors rather than continuous progress modeling. In contrast, GPT-5 and ProgressLM (3B-SFT and 3B-RL) produce smoother distributions, reflecting improved sensitivity to intermediate task progress.