Triangle Splatting for Real-Time Radiance Field Rendering

TL;DR

Triangle Splatting presents a differentiable renderer that directly optimizes unstructured triangles, reconciling the efficiency of traditional triangle pipelines with end-to-end gradient-based optimization. A novel window function based on a projected triangle's signed-distance field enables precise, bounded, and differentiable blending of triangle contributions. The approach includes adaptive pruning and densification to manage representation capacity and an SfM-based initialization with a comprehensive loss to drive geometry and appearance toward photo-realistic novel-view synthesis. Results show superior perceptual quality and competitive or superior speed compared with non-volumetric primitives and state-of-the-art methods, with strong indoor performance and full compatibility with mesh-based renderers. This work bridges classic graphics and differentiable rendering, enabling real-time, mesh-ready radiance-field reconstruction and rendering with triangles as the fundamental primitive.

Abstract

The field of computer graphics was revolutionized by models such as Neural Radiance Fields and 3D Gaussian Splatting, displacing triangles as the dominant representation for photogrammetry. In this paper, we argue for a triangle comeback. We develop a differentiable renderer that directly optimizes triangles via end-to-end gradients. We achieve this by rendering each triangle as differentiable splats, combining the efficiency of triangles with the adaptive density of representations based on independent primitives. Compared to popular 2D and 3D Gaussian Splatting methods, our approach achieves higher visual fidelity, faster convergence, and increased rendering throughput. On the Mip-NeRF360 dataset, our method outperforms concurrent non-volumetric primitives in visual fidelity and achieves higher perceptual quality than the state-of-the-art Zip-NeRF on indoor scenes. Triangles are simple, compatible with standard graphics stacks and GPU hardware, and highly efficient: for the \textit{Garden} scene, we achieve over 2,400 FPS at 1280x720 resolution using an off-the-shelf mesh renderer. These results highlight the efficiency and effectiveness of triangle-based representations for high-quality novel view synthesis. Triangles bring us closer to mesh-based optimization by combining classical computer graphics with modern differentiable rendering frameworks. The project page is https://trianglesplatting.github.io/

Figures (12)

• Figure 1: We propose a new representation for differentiable rendering based on the most classical of 3D primitives: the triangle. We show how a triangle soup (i.e. unstructured, disconnected triangles) can be optimized effectively, generating state-of-the-art novel view synthesis images while being immediately compatible with classical rendering pipelines. The figure shows the final rendered output (left), a visualization of soft blending (middle), and the rendering of a random subset of triangles to highlight their structure (right).
• Figure 2: Byproduct of the triangle-based representation -- Triangle Splatting unifies differentiable scene optimization with traditional graphics pipelines. The optimized triangle soup is compatible with any mesh-based renderer, enabling seamless integration into traditional graphics pipelines. In a game engine, we render at 2,400+ FPS at 1280×720 resolution on an RTX4090 (left) and 300+ FPS on a consumer laptop (right).
• Figure 3: Triangle window function (1D and 2D) -- We visualize the window functions of prior works Held20253DConvexDeng2020CvxNet (bottom) vs. the one introduced in our paper (top) in both 1D (left) and 2D (right). We show how the window function changes as we vary the smoothness control parameter $\sigma$. As $\sigma$ decreases, note that both can approximate the window function of a triangle. However, as $\sigma$ increases, the support of \ref{['eq:indicator2']} exceeds the footprint of the triangle, making it unsuitable for rasterization workloads. In the limit, \ref{['eq:indicator2']} becomes globally supported, with a window value of 0.5 everywhere, causing every triangle to contribute to the color of every pixel in the image.
• Figure 4: Triangle prunning -- To reduce floaters, we prune triangles seen in fewer than two views with more than one pixel of coverage, removing those that are overfitted by a single training view.
• Figure 5: Limitations of PSNR -- Due to its inherent smoothness, the Gaussian primitive tends to perform better on the PSNR metric, which evaluates pixel-wise differences, despite being blurrier. In the highlighted region, our method (TS) achieves a PSNR of 18.41, compared to 21.27 for 2DGS.