Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A3R: Agentic Affordance Reasoning via Cross-Dimensional Evidence in 3D Gaussian Scenes

Di Li, Jie Feng, Guanbin Li, Ronghua Shang, Yuhui Zheng, Weisheng Dong, Guangming Shi

Abstract

Affordance reasoning in 3D Gaussian scenes aims to identify the region that supports the action specified by a given text instruction in complex environments. Existing methods typically cast this problem as one-shot prediction from static scene observations, assuming sufficient evidence is already available for reasoning. However, in complex 3D scenes, many failure cases arise not from weak prediction capacity, but from incomplete task-relevant evidence under fixed observations. To address this limitation, we reformulate fine-grained affordance reasoning as a sequential evidence acquisition process, where ambiguity is progressively reduced through complementary 3D geometric and 2D semantic evidence. Building on this formulation, we propose A3R, an agentic affordance reasoning framework that enables an MLLM-based policy to iteratively select evidence acquisition actions and update the affordance belief through cross-dimensional evidence acquisition. To optimize such sequential decision making, we further introduce a GRPO-based policy learning strategy that improves evidence acquisition efficiency and reasoning accuracy. Extensive experiments on scene-level benchmarks show that A3R consistently surpasses static one-shot baselines, demonstrating the advantage of agentic cross-dimensional evidence acquisition for fine-grained affordance reasoning in complex 3D Gaussian scenes.

A3R: Agentic Affordance Reasoning via Cross-Dimensional Evidence in 3D Gaussian Scenes

Abstract

Affordance reasoning in 3D Gaussian scenes aims to identify the region that supports the action specified by a given text instruction in complex environments. Existing methods typically cast this problem as one-shot prediction from static scene observations, assuming sufficient evidence is already available for reasoning. However, in complex 3D scenes, many failure cases arise not from weak prediction capacity, but from incomplete task-relevant evidence under fixed observations. To address this limitation, we reformulate fine-grained affordance reasoning as a sequential evidence acquisition process, where ambiguity is progressively reduced through complementary 3D geometric and 2D semantic evidence. Building on this formulation, we propose A3R, an agentic affordance reasoning framework that enables an MLLM-based policy to iteratively select evidence acquisition actions and update the affordance belief through cross-dimensional evidence acquisition. To optimize such sequential decision making, we further introduce a GRPO-based policy learning strategy that improves evidence acquisition efficiency and reasoning accuracy. Extensive experiments on scene-level benchmarks show that A3R consistently surpasses static one-shot baselines, demonstrating the advantage of agentic cross-dimensional evidence acquisition for fine-grained affordance reasoning in complex 3D Gaussian scenes.

Paper Structure

This paper contains 19 sections, 27 equations, 3 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. 3D Affordance Reasoning
  4. 2D--3D Semantic Transfer for Scene Understanding
  5. Active Perception and Agentic Reasoning
  6. Method
  7. Formulation as a Partially Observable Markov Decision Process
  8. Cross-Dimensional Evidence Acquisition Operators
  9. MLLM-Driven Agentic Policy
  10. Scene-Grounded Decision State Transition
  11. GRPO-Based Policy Learning
  12. Experiments
  13. Experimental Setup
  14. Main Results
  15. Comparison of Evidence Acquisition Strategies
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Overview of the proposed framework. The agent maintains a scene-grounded decision state composed of the current workspace, affordance belief, and action--evidence history. Conditioned on this state, the policy selects 2D or 3D evidence operators whose outputs are projected back to the scene support to update the state, progressively reducing ambiguity for fine-grained affordance reasoning in 3DGS scenes.
  • Figure 2: Qualitative comparison with static baselines. Our method produces more accurate and spatially precise affordance predictions, especially in ambiguity-heavy scenes that require fine-grained localization and distractor suppression.
  • Figure 3: Visualization of evidence-acquisition trajectories. The agent adaptively selects 2D semantic and 3D geometric operators based on the current belief, progressively refining the affordance prediction through coarse-to-fine cross-dimensional evidence acquisition.