PUFM++: Point Cloud Upsampling via Enhanced Flow Matching

TL;DR

PUFM++ advances point cloud upsampling by introducing a two-stage flow-matching framework with a patch-based local transport strategy, an adaptive time scheduler for efficient inference, and manifold-aware post-processing. A latent-state recurrent velocity estimator (RIN) endows the model with memory across steps, improving global shape consistency and edge sharpness. Extensive experiments on synthetic and real-world data show state-of-the-art fidelity, density uniformity, and mesh quality, with robust performance under noise and partial observations. The work delivers practical impact for 3D perception tasks and downstream applications while addressing ethical and environmental considerations through efficient design and transparent reporting.

Abstract

Recent advances in generative modeling have demonstrated strong promise for high-quality point cloud upsampling. In this work, we present PUFM++, an enhanced flow-matching framework for reconstructing dense and accurate point clouds from sparse, noisy, and partial observations. PUFM++ improves flow matching along three key axes: (i) geometric fidelity, (ii) robustness to imperfect input, and (iii) consistency with downstream surface-based tasks. We introduce a two-stage flow-matching strategy that first learns a direct, straight-path flow from sparse inputs to dense targets, and then refines it using noise-perturbed samples to approximate the terminal marginal distribution better. To accelerate and stabilize inference, we propose a data-driven adaptive time scheduler that improves sampling efficiency based on interpolation behavior. We further impose on-manifold constraints during sampling to ensure that generated points remain aligned with the underlying surface. Finally, we incorporate a recurrent interface network~(RIN) to strengthen hierarchical feature interactions and boost reconstruction quality. Extensive experiments on synthetic benchmarks and real-world scans show that PUFM++ sets a new state of the art in point cloud upsampling, delivering superior visual fidelity and quantitative accuracy across a wide range of tasks. Code and pretrained models are publicly available at https://github.com/Holmes-Alan/Enhanced_PUFM.

Figures (15)

• Figure 1: Flow Matching for Point Cloud Upsampling. We extract paired patches from sparse and dense point clouds and learn the velocity field for point cloud upsampling.
• Figure 2: Overview of the proposed iterative point cloud upsampling network. The network takes the current sampled point cloud $\mathbf{x}_t$, sampling step $t$, and previous latent code $z$ as input. A point feature encoder extracts features $f(\mathbf{x}_t)$. The latent interface $z_t$ is initialized by conditioning on time and global embedding. The core processing consists of stacked RIN blocks. The network outputs the estimated velocity field $\nu_\theta$ and the updated latent code $z_{t+1}$ for the next iteration.
• Figure 3: Visual comparison of different methods on 4-times upsampling. We apply different methods to four examples from the PU1K dataset. Our method generates evenly distributed points and sharp edges, while others exhibit obvious holes and noisy surfaces (see the enlarged region in red boxes).
• Figure 4: Visual comparison of different methods on 16-times upsampling for mesh reconstruction. We apply different methods to five examples from the PUGAN and PU1K dataset. Then we use the Marching Cubes algorithm to reconstruct meshes from the upsampled point clouds for qualitative comparison.
• Figure 5: Visual comparison of training with/without pre-alignment and refinement. The final PUFM++ (4th column) is compared to: PUFM++ without pre-alignment/refinement (1st), with CD pre-alignment only (2nd), and with EMD pre-alignment only (3rd).