IPCD: Intrinsic Point-Cloud Decomposition

TL;DR

IPCD addresses the problem of decomposing colored point clouds into intrinsic components by extending intrinsic image decomposition to 3D data. The proposed IPCD-Net combines point-wise feature aggregation with Projection-based Luminance Distribution (PLD) to capture global-light cues, augmented by a hierarchical refinement and a shared encoder. Empirical results on a synthetic outdoor IPCD dataset show reduced cast shadows and improved shade color fidelity, with successful demonstrations in texture editing, relighting, and registration under varying illumination, and robustness on real-world scenes without per-scene fine-tuning. This work enables direct, physically-grounded manipulation of 3D appearance and lighting in point clouds, with potential for broader applications in AR, robotics, and 3D scene understanding.

Abstract

Point clouds are widely used in various fields, including augmented reality (AR) and robotics, where relighting and texture editing are crucial for realistic visualization. Achieving these tasks requires accurately separating albedo from shade. However, performing this separation on point clouds presents two key challenges: (1) the non-grid structure of point clouds makes conventional image-based decomposition models ineffective, and (2) point-cloud models designed for other tasks do not explicitly consider global-light direction, resulting in inaccurate shade. In this paper, we introduce \textbf{Intrinsic Point-Cloud Decomposition (IPCD)}, which extends image decomposition to the direct decomposition of colored point clouds into albedo and shade. To overcome challenge (1), we propose \textbf{IPCD-Net} that extends image-based model with point-wise feature aggregation for non-grid data processing. For challenge (2), we introduce \textbf{Projection-based Luminance Distribution (PLD)} with a hierarchical feature refinement, capturing global-light ques via multi-view projection. For comprehensive evaluation, we create a synthetic outdoor-scene dataset. Experimental results demonstrate that IPCD-Net reduces cast shadows in albedo and enhances color accuracy in shade. Furthermore, we showcase its applications in texture editing, relighting, and point-cloud registration under varying illumination. Finally, we verify the real-world applicability of IPCD-Net.

Figures (8)

• Figure 1: Motivation of Intrinsic Point-Cloud Decomposition (IPCD). (a) Directly using colored point clouds for texture editing and relighting introduces artifacts due to illumination, resulting in unnatural colors (green) and residual shade (blue). To address this issue, (b) we extend image decomposition to point clouds, namely IPCD, which decomposes point clouds into albedo and shade. Applying IPCD enables more natural texture editing and significantly reduces residual shade in relighting.
• Figure 2: Architectures of (a) IPCD-Net$_{\text{Base}}$ and (b) IPCD-Net. IPCD-Net$_{\text{Base}}$ independently estimates albedo $\hat{A}$ and shade $\hat{S}$ from the colored point cloud $I$ using separate albedo and shade estimators. IPCD-Net employs a shared feature extractor to process $I$, generating pre-albedo and pre-shade representations ($\hat{A'}, \hat{S'}$). The Projection-based Luminance Distribution (PLD) is processed by SphereNet to capture global-light features and enhance IPCD performance. The final albedo $\hat{A}$ and shade $\hat{S}$ are obtained by integrating PLD features with the pre-albedo and pre-shade representations through a hierarchical feature refinement.
• Figure 3: PLD overview. The input point cloud is first projected onto an image plane from a specific viewpoint. By systematically varying the projection angles and computing the average luminance across these views, the PLD is constructed and represented on a hemispherical surface. Projections aligned with the illumination direction result in brighter images with minimal shade, whereas projections from the opposite direction yield darker images dominated by shadowed regions. The color bar represents the luminance intensity from its corresponding view.
• Figure 4: Visual comparison of IPCD performance. Retinex grosse2009 exhibits residual cast shadows, while more recent models mitigate this issue. IID-Anything chen2025intrinsicanything performs IPCD through point-cloud rendering, but inconsistencies in albedo brightness across rendered images introduce noise, and dimensional reduction during rendering leads to remaining shade. Our IPCD models effectively separate albedo from shade. However, IPCD-Net$_{\text{Base}}$ lacks explicit light direction, causing some cast shadows to be omitted in shade, as indicated by red arrows. Conversely, IPCD-Net successfully estimates the shade color and cast shadows.
• Figure 5: Visual result of texture editing with IPCD. (a) Input point cloud, (b) direct editing, and (c) editing through IPCD-Net are shown. Direct texture editing results in unnatural appearance due to lighting inconsistencies. Editing albedo separated by IPCD-Net, then recombining with shade, achieving natural results.