EdgeGaussians -- 3D Edge Mapping via Gaussian Splatting

TL;DR

EdgeGaussians replaces implicit neural edge fields with explicit oriented 3D edge points modeled as Gaussians, learning each point's position via its mean $\mu$ and its edge direction via the principal axis of covariance $\Sigma$ with $G(x) = \exp(-\tfrac{1}{2}(x-\mu)^T \Sigma^{-1}(x-\mu))$ and enforcing $\Sigma = R S S^{\top} R^{\top}$. Training uses rendering-based losses and two geometric regularizers: $\mathcal{L}_{\text{orient}} = 1 - \frac{1}{N} \left( \frac{1}{k} \sum_{i=1}^{N} \sum_{j=1}^{k} |d_i^{\top} d_{i_j}| \right)$ and $\mathcal{L}_{\text{shape}} = \frac{1}{N} \sum_{i=1}^{N} \frac{{}^{2}s_i}{{}^{1}s_i}$, while supervision comes from 2D edge maps via a masked projection loss $\mathcal{L}_{\text{proj}}$. The learned Gaussians are clustered into edges by a graph traversal that enforces spatial proximity and orientation consistency, then fitted with either lines or cubic Bézier curves, selected by comparing curve vs. line residuals with a threshold $\delta$. Empirically, EdgeGaussians matches or exceeds state-of-the-art accuracy and completeness while offering an order-of-magnitude faster training than prior implicit methods, demonstrating practical benefits for fast 3D edge mapping from multi-view data.

Abstract

With their meaningful geometry and their omnipresence in the 3D world, edges are extremely useful primitives in computer vision. 3D edges comprise of lines and curves, and methods to reconstruct them use either multi-view images or point clouds as input. State-of-the-art image-based methods first learn a 3D edge point cloud then fit 3D edges to it. The edge point cloud is obtained by learning a 3D neural implicit edge field from which the 3D edge points are sampled on a specific level set (0 or 1). However, such methods present two important drawbacks: i) it is not realistic to sample points on exact level sets due to float imprecision and training inaccuracies. Instead, they are sampled within a range of levels so the points do not lie accurately on the 3D edges and require further processing. ii) Such implicit representations are computationally expensive and require long training times. In this paper, we address these two limitations and propose a 3D edge mapping that is simpler, more efficient, and preserves accuracy. Our method learns explicitly the 3D edge points and their edge direction hence bypassing the need for point sampling. It casts a 3D edge point as the center of a 3D Gaussian and the edge direction as the principal axis of the Gaussian. Such a representation has the advantage of being not only geometrically meaningful but also compatible with the efficient training optimization defined in Gaussian Splatting. Results show that the proposed method produces edges as accurate and complete as the state-of-the-art while being an order of magnitude faster. Code is released at https://github.com/kunalchelani/EdgeGaussians.

Figures (11)

• Figure 1: EdgeGaussians: the proposed method learns oriented 3D edge points via Gaussian Splatting specialized for 2D edge maps. The mean and the direction of largest variance of a Gaussian define an edge point's position and orientation. Left to right: 2D edge maps generated by off-the-shelf detectors su2021pixelpoma2020dense are used as supervisory signals to train the 3D edge Gaussians. The trained Gaussians are clustered based on spatial proximity and orientation consistency. Parametric edges are fitted on top of these clusters.
• Figure 2: Occlusions for 3D edges. The red edges are the edges occluded by surface and are absent from the supervisory 2D edge maps. Yet, these edges are present in the rendering of the 3D edge representation, which is the desired behavior.
• Figure 3: Geometric regularization of 3D Gaussians. Enforcing elliptical Gaussians with the principal direction aligned with the neighbors would result in 2D projections as shown on the left, while un-regularized ones might look like those on the right. The ones on the left are geometrically meaningful and result in easier clustering into separate edges.
• Figure 4: Qualitative results on ABC-NEFYe_2023_CVPR. The proposed method captures curves and lines accurately but can be marginally incomplete. EMAP Li2024CVPR is either slightly less complete than the proposed method or predicts extra edges. NEF Ye_2023_CVPR produces correct edges but exhibits knots observed around corners on many occasions.
• Figure 5: Qualitative results on DTU jensen2014large. Comparison of the proposed 3D reconstruction method against baselines. NEAT xue2024neat produces partially complete reconstructions with lines only. The edges from EMAP Li2024CVPR are more complete but there are duplicate edge predictions for a single target edge in 3D, while our reconstruction is much cleaner with mostly a single predicted edge boundary (see zoom-ins).