3DGS-to-PC: Convert a 3D Gaussian Splatting Scene into a Dense Point Cloud or Mesh

TL;DR

The paper tackles the challenge of visualizing and processing 3D Gaussian Splatting (3DGS) scenes by converting them into dense point clouds and meshes compatible with standard 3D pipelines. It proposes 3DGS-to-PC, a configurable workflow that probabilistically samples points from Gaussian volumes with volume-based allocation and re-colours points according to their contributions to rendered images, using a Mahalanobis distance threshold $D_M$ to remove outliers ($D_M(x) = \sqrt{(\mathbf{x}-\boldsymbol{\mu})^\top \mathbf{\Sigma}^{-1} (\mathbf{x}-\boldsymbol{\mu})}$, with a default of 2). Gaussian colours are updated to reflect view-dependent appearance by evaluating per-pixel contributions across camera poses, ensuring colours align with the rendered scene rather than raw Gaussian colours. Meshes are produced by identifying surface Gaussians and applying Poisson Surface Reconstruction to a cleaned surface-focused point cloud, with optional Laplacian smoothing. The framework is highly compatible with existing 3DGS pipelines, enables dense representations without retraining, and highlights future work to speed up rendering and reduce reliance on pre-existing camera poses.

Abstract

3D Gaussian Splatting (3DGS) excels at producing highly detailed 3D reconstructions, but these scenes often require specialised renderers for effective visualisation. In contrast, point clouds are a widely used 3D representation and are compatible with most popular 3D processing software, yet converting 3DGS scenes into point clouds is a complex challenge. In this work we introduce 3DGS-to-PC, a flexible and highly customisable framework that is capable of transforming 3DGS scenes into dense, high-accuracy point clouds. We sample points probabilistically from each Gaussian as a 3D density function. We additionally threshold new points using the Mahalanobis distance to the Gaussian centre, preventing extreme outliers. The result is a point cloud that closely represents the shape encoded into the 3D Gaussian scene. Individual Gaussians use spherical harmonics to adapt colours depending on view, and each point may contribute only subtle colour hints to the resulting rendered scene. To avoid spurious or incorrect colours that do not fit with the final point cloud, we recalculate Gaussian colours via a customised image rendering approach, assigning each Gaussian the colour of the pixel to which it contributes most across all views. 3DGS-to-PC also supports mesh generation through Poisson Surface Reconstruction, applied to points sampled from predicted surface Gaussians. This allows coloured meshes to be generated from 3DGS scenes without the need for re-training. This package is highly customisable and capability of simple integration into existing 3DGS pipelines. 3DGS-to-PC provides a powerful tool for converting 3DGS data into point cloud and surface-based formats.

Figures (7)

• Figure 1: Showcase of how 3DGS-to-PC can convert the bike scene from the Mip-NeRF 360 abs-2111-12077 dataset from a 3D Gaussian splat into a dense point cloud.
• Figure 2: Comparison between the different techniques for generating point clouds of the Mip-NeRF 360 bike scene. The left shows a point cloud where the colours of each point are the colours of the Gaussians that they have been generated from. The right shows a point cloud where the colours have been generated via our colour rendering process. These point cloud are shown in the CloudCompare viewer, and all points were set to a size of 3 to show the effect of the noise produced from using colours from the original Gaussians.
• Figure 3: Demonstration of how the Gaussian colours are determined based on rendered colours in a scene. The scene consists of a set of Gaussians that make up the surface of a yellow object, with a seperate gray background surface. The transparency (alpha) values of each Gaussian are shown. The left side of the diagram demonstrates pixel colour rendering, where occluded Gaussians (e.g. green and red) contribute collectively to the final pixel colour, even though individually these do not match the environment. The right side illustrates the update process, where each Gaussian's colour is adjusted to match the pixel it contributed to the most. For instance, the green Gaussian is updated to reflect the yellow pixel rather than the gray one, since the gray Gaussian is opaque. In practice, this process involves more cameras and pixels than depicted in the simplified diagram.
• Figure 4: Demonstration of point generation from Gaussians with rendered colours. The number of points generated for each Gaussian is determined by its volume, calculated using Equation \ref{['eq:gaussian_volume']}, which is shown on the left of the diagram. Distances between the Gaussian centre and generated points are evaluated using Equation \ref{['eq:mahalanobis']}. Points exceeding a distance of 2 standard deviation are removed and regenerated, as shown on the right of the diagram. This iterative process ensures that the final point cloud accurately represents the scene, with the number of points per Gaussian being proportional to its volume.
• Figure 5: Demonstration of how this framework generates meshes from a point cloud. The process begins with identifying surface Gaussians using the Gaussian renderer, selecting only those with the highest contribution in each rendered image. Points are then sampled from these surface Gaussians to represent the scene's surfaces. The point normals are then calculated by taking the perpendicular vector of surface on the smallest side of each Gaussian. Finally, Poisson Surface Reconstruction connects the sampled points to generate the final mesh.