Radiant Triangle Soup with Soft Connectivity Forces for 3D Reconstruction and Novel View Synthesis

TL;DR

The paper presents Radiant Triangle Soup (RTS), a triangle-based scene representation for gradient-based optimization of 3D geometry and appearance, addressing limitations of Gaussian splatting by enabling sharp boundaries and explicit inter-primitive communication. RTS introduces diffuse triangle boundaries, per-vertex Spherical Harmonics, and a differentiable rasterization pipeline, augmented by soft connectivity losses and adaptive density control to align primitives into coherent surfaces. Through extensive experiments on DTU and mip-NeRF 360, RTS achieves competitive geometric accuracy and strong novel-view synthesis, with ablative evidence showing the importance of connectivity and depth supervision. The approach offers a compact, expressive representation that directly coordinates neighboring primitives, enabling improved surface fidelity while maintaining rendering performance, though it currently cannot produce watertight meshes and incurs some runtime overhead from dynamic connectivity maintenance.

Abstract

We introduce an inference-time scene optimization algorithm utilizing triangle soup, a collection of disconnected translucent triangle primitives, as the representation for the geometry and appearance of a scene. Unlike full-rank Gaussian kernels, triangles are a natural, locally-flat proxy for surfaces that can be connected to achieve highly complex geometry. When coupled with per-vertex Spherical Harmonics (SH), triangles provide a rich visual representation without incurring an expensive increase in primitives. We leverage our new representation to incorporate optimization objectives and enforce spatial regularization directly on the underlying primitives. The main differentiator of our approach is the definition and enforcement of soft connectivity forces between triangles during optimization, encouraging explicit, but soft, surface continuity in 3D. Experiments on representative 3D reconstruction and novel view synthesis datasets show improvements in geometric accuracy compared to current state-of-the-art algorithms without sacrificing visual fidelity.

Figures (11)

• Figure 1: Optimizing Radiant Triangle Soup (RTS) produces high quality 3D models from captured images. Above, from left to right, are the rendered normal map, rendered depth map, rendered image, and the direct rasterization of the triangle primitives, as if they were opaque.
• Figure 2: We begin with a set of sparse points. From these points we initialize a set of triangles and compute nearest neighbor connections for each triangle edge. During optimization, we render an image, depth map, and normal map for each view. We compute both 2D loss (over all output renderings), as well as 3D loss (directly over the primitive connections). Adaptive densification is non-differentiable and is performed at set intervals throughout the optimization. Triangle edge neighbors are recomputed following adaptive densification.
• Figure 3: Triangle parameterization. Each triangle is parameterized by the incenter $\mu$, and three scale values $s^0s^1s^2$. These parameters, along with the rotation matrix $R$, define the coordinates of each vertex $V^j$.
• Figure 4: Controlling triangle diffuseness with $\sigma$. Small values of $\sigma$(left) result in diffuse triangles with extended range of influence. Large values of $\sigma$(right) result in sharp triangles with no blurring across edges.
• Figure 5: Examples of edge association for connecting nearby triangles. For simplicity, it is assumed the triangles are co-planar in this figure.