MICAS: Multi-grained In-Context Adaptive Sampling for 3D Point Cloud Processing

TL;DR

MICAS tackles inter-task and intra-task sensitivity in 3D point cloud in-context learning by introducing a dual-module framework: task-adaptive point sampling to align point-level sampling with task prompts via a differentiable Gumbel mechanism, and query-specific prompt sampling to select the most effective demonstration prompts for each query using pseudo-labels and listwise ranking. The approach integrates with a patch-based ICL model to produce robust cross-task performance across reconstruction, denoising, registration, and part segmentation, achieving notable gains (e.g., a 4.1% improvement in part segmentation) over previous methods. Extensive experiments on ShapeNet In-Context Dataset demonstrate MICAS’s superior adaptability and robustness, with ablations confirming complementary benefits of the two modules. The work highlights a practical advancement for adaptive in-context learning in 3D PCP, enabling a single model to handle diverse tasks with improved reliability and efficiency.

Abstract

Point cloud processing (PCP) encompasses tasks like reconstruction, denoising, registration, and segmentation, each often requiring specialized models to address unique task characteristics. While in-context learning (ICL) has shown promise across tasks by using a single model with task-specific demonstration prompts, its application to PCP reveals significant limitations. We identify inter-task and intra-task sensitivity issues in current ICL methods for PCP, which we attribute to inflexible sampling strategies lacking context adaptation at the point and prompt levels. To address these challenges, we propose MICAS, an advanced ICL framework featuring a multi-grained adaptive sampling mechanism tailored for PCP. MICAS introduces two core components: task-adaptive point sampling, which leverages inter-task cues for point-level sampling, and query-specific prompt sampling, which selects optimal prompts per query to mitigate intra-task sensitivity. To our knowledge, this is the first approach to introduce adaptive sampling tailored to the unique requirements of point clouds within an ICL framework. Extensive experiments show that MICAS not only efficiently handles various PCP tasks but also significantly outperforms existing methods. Notably, it achieves a remarkable $4.1\%$ improvement in the part segmentation task and delivers consistent gains across various PCP applications.

Figures (6)

• Figure 1: Inter-task and intra-task sensitivities in in-context learning. The red and green points are sampled using Farthest Point Sampling (FPS) and task-adaptive point sampling, respectively. The blue circles indicate erroneous sampling points, while the red ovals highlight missing points in the predicted point cloud, caused by the absence of a central point within the region. (Zoom in for more details)
• Figure 2: Comparison between the proposed MICAS and the traditional in-context learning framework.
• Figure 3: Overview of the proposed MAL-ICL. (a) Task-adaptive point sampling is designed to achieve better point-level sampling. (b) Query-specific prompt sampling aims to infer the most effective prompt-level sampling.
• Figure 4: Qualitative experimental results compared with the PIC-Cat fang2024explore and PIC-Sep fang2024explore. The red ovals represent the difference between the two methods. Additional visualization results can be found in the supplementary material. (Zoom in for more details)
• Figure A1: Qualitative experimental results compared with the PIC-Cat fang2024explore. The red and green points denote the central points selected by PIC-Cat and our proposed MICAS, respectively. (Zoom in for more details)