Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PFGS: High Fidelity Point Cloud Rendering via Feature Splatting

Jiaxu Wang, Ziyi Zhang, Junhao He, Renjing Xu

TL;DR

This work introduces PFGS (Point Feature Gaussian Splatting), a high-fidelity point-cloud renderer that bridges explicit point geometry with 3D Gaussian Splatting to render photorealistic views from sparse data. The pipeline consists of a multiscale feature extractor, a Gaussian regressor, and a multiscale, recurrent decoder that progressively fuses Gaussian splatting with neural feature splatting, guided by a two-stage training regime and combined losses that promote both spatial accuracy and high-frequency details. Key contributions include a lightweight, 3D-aware rendering framework that achieves superior rendering quality across indoor, object, and human datasets without per-scene optimization, and comprehensive ablations demonstrating the necessity of progressive, multiscale decoding and the complementary roles of Gaussian and feature splatting. The approach advances real-time, consistent 3D-aware view synthesis from sparse point clouds, with practical impact for VR/AR, robotics, and autonomous systems where dense data may be unavailable.

Abstract

Rendering high-fidelity images from sparse point clouds is still challenging. Existing learning-based approaches suffer from either hole artifacts, missing details, or expensive computations. In this paper, we propose a novel framework to render high-quality images from sparse points. This method first attempts to bridge the 3D Gaussian Splatting and point cloud rendering, which includes several cascaded modules. We first use a regressor to estimate Gaussian properties in a point-wise manner, the estimated properties are used to rasterize neural feature descriptors into 2D planes which are extracted from a multiscale extractor. The projected feature volume is gradually decoded toward the final prediction via a multiscale and progressive decoder. The whole pipeline experiences a two-stage training and is driven by our well-designed progressive and multiscale reconstruction loss. Experiments on different benchmarks show the superiority of our method in terms of rendering qualities and the necessities of our main components.

PFGS: High Fidelity Point Cloud Rendering via Feature Splatting

TL;DR

This work introduces PFGS (Point Feature Gaussian Splatting), a high-fidelity point-cloud renderer that bridges explicit point geometry with 3D Gaussian Splatting to render photorealistic views from sparse data. The pipeline consists of a multiscale feature extractor, a Gaussian regressor, and a multiscale, recurrent decoder that progressively fuses Gaussian splatting with neural feature splatting, guided by a two-stage training regime and combined losses that promote both spatial accuracy and high-frequency details. Key contributions include a lightweight, 3D-aware rendering framework that achieves superior rendering quality across indoor, object, and human datasets without per-scene optimization, and comprehensive ablations demonstrating the necessity of progressive, multiscale decoding and the complementary roles of Gaussian and feature splatting. The approach advances real-time, consistent 3D-aware view synthesis from sparse point clouds, with practical impact for VR/AR, robotics, and autonomous systems where dense data may be unavailable.

Abstract

Rendering high-fidelity images from sparse point clouds is still challenging. Existing learning-based approaches suffer from either hole artifacts, missing details, or expensive computations. In this paper, we propose a novel framework to render high-quality images from sparse points. This method first attempts to bridge the 3D Gaussian Splatting and point cloud rendering, which includes several cascaded modules. We first use a regressor to estimate Gaussian properties in a point-wise manner, the estimated properties are used to rasterize neural feature descriptors into 2D planes which are extracted from a multiscale extractor. The projected feature volume is gradually decoded toward the final prediction via a multiscale and progressive decoder. The whole pipeline experiences a two-stage training and is driven by our well-designed progressive and multiscale reconstruction loss. Experiments on different benchmarks show the superiority of our method in terms of rendering qualities and the necessities of our main components.
Paper Structure (31 sections, 13 equations, 11 figures, 8 tables)

This paper contains 31 sections, 13 equations, 11 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Point Cloud Render
  4. Neural-based View Synthesis
  5. Methodology
  6. Preliminaries
  7. Gaussian Feature Prediction with Multiscale Feature Extraction
  8. Multiscale and Progressive Feature Decoding
  9. Training Strategies
  10. Experiments
  11. Experiments setup
  12. Comparisons With Other Point Cloud Renderers.
  13. Ablation studies.
  14. Ablations of main components.
  15. Analysis of Gaussian and Feature Splatting.
  16. ...and 16 more sections

Figures (11)

  • Figure 1: The main pipeline of the proposed approach. The above panel includes the multiscale feature extractor and Gaussian regressor. Both Gaussian and features are splatted to the 2D plane and concatenated and fed to the progressive and multiscale feature decoding module described in the low panel.
  • Figure 2: Qualitative comparisons between ours and other point cloud renderers on the ScanNet dataset.
  • Figure 3: Qualitative comparisons between ours and other point cloud rendering methods on the DTU dataset.
  • Figure 4: Qualitative comparisons between ours and other point cloud renderers on the THuman2.0 dataset.
  • Figure 5: Qualitative Comparisons of the renderings produced by the Gaussian splatting after the first stage and the feature splatting after the whole framework.
  • ...and 6 more figures