DisC-GS: Discontinuity-aware Gaussian Splatting

TL;DR

A novel framework enabling Gaussian Splatting to perform discontinuity-aware image rendering is proposed and a Bezier-boundary gradient approximation strategy is introduced within this framework to keep the "differentiability" of the proposed discontinuity-aware rendering process.

Abstract

Recently, Gaussian Splatting, a method that represents a 3D scene as a collection of Gaussian distributions, has gained significant attention in addressing the task of novel view synthesis. In this paper, we highlight a fundamental limitation of Gaussian Splatting: its inability to accurately render discontinuities and boundaries in images due to the continuous nature of Gaussian distributions. To address this issue, we propose a novel framework enabling Gaussian Splatting to perform discontinuity-aware image rendering. Additionally, we introduce a Bézier-boundary gradient approximation strategy within our framework to keep the "differentiability" of the proposed discontinuity-aware rendering process. Extensive experiments demonstrate the efficacy of our framework.

Figures (2)

• Figure 1: (a) Illustration of a ground truth image, containing numerous discontinuities and boundaries, that is expected to be rendered from a certain viewpoint of a 3D scene. We generate the boundary map in (a) utilizing the Canny algorithm canny1986computational. (b) Illustration of Gaussian distributions projected onto the image plane. As shown, since Gaussian distributions are continuous, they can inevitably "pass over" the (hard) boundary represented by the curve. (c) Illustration of images rendered with and without applying DisC-GS. As shown, without DisC-GS, Gaussian Splatting can fail to accurately render boundaries. In contrast, applying DisC-GS ensures that boundaries and discontinuities in the image are properly rendered. More qualitative results are in Supplementary. (Best viewed in color.)
• Figure 2: Illustration of the discontinuity-aware rendering process over a single Gaussian distribution. Specifically, over each 2D Gaussian distribution representing the 3D scene, we first introduce it with a new attribute $c_{curve} \in \mathbb{R}^{4M \times 2}$ (represented by the red and purple points in (a)). Here we set $M = 2$. After that, given a viewpoint, as shown in (b), we project both the Gaussian distribution and the points stored in $c_{curve}$ onto the image plane corresponding to the viewpoint. Finally, leveraging the modified $\alpha$-blending function in Eq. \ref{['eq:method_3']}, we can perform discontinuity-aware rendering and render only the part of the Gaussian distribution masked with the dotted lines in (c). (Best viewed in color.)