MeshSplatting: Differentiable Rendering with Opaque Meshes

TL;DR

<3-5 sentence high-level summary>MeshSplatting tackles the gap between neural mesh optimization and real-time mesh-based engines by directly optimizing a connected, opaque triangle mesh through differentiable rendering. It starts from an unstructured triangle soup derived from SfM and progressively converts it into a coherent mesh via restricted Delaunay triangulation, while jointly optimizing geometry, color, and opacity. The method achieves higher visual fidelity and significantly lower training time and memory than state-of-the-art mesh-based approaches, and outputs directly usable meshes for game engines, physics, and ray tracing. This work thus bridges neural rendering and interactive graphics, enabling real-time scene interaction, object extraction, and downstream applications without post-processing.

Abstract

Primitive-based splatting methods like 3D Gaussian Splatting have revolutionized novel view synthesis with real-time rendering. However, their point-based representations remain incompatible with mesh-based pipelines that power AR/VR and game engines. We present MeshSplatting, a mesh-based reconstruction approach that jointly optimizes geometry and appearance through differentiable rendering. By enforcing connectivity via restricted Delaunay triangulation and refining surface consistency, MeshSplatting creates end-to-end smooth, visually high-quality meshes that render efficiently in real-time 3D engines. On Mip-NeRF360, it boosts PSNR by +0.69 dB over the current state-of-the-art MiLo for mesh-based novel view synthesis, while training 2x faster and using 2x less memory, bridging neural rendering and interactive 3D graphics for seamless real-time scene interaction. The project page is available at https://meshsplatting.github.io/.

Figures (13)

• Figure 1: MeshSplatting produces a connected mesh composed only of opaque triangles, achieving high-quality novel view synthesis through end-to-end optimization, with a $2\times$ training speed-up and $2\times$ lower memory usage over current state-of-the-art methods. (a) Our representation is compatible with standard game engines, requiring no a-posteriori conversion and/or custom rendering routines for transparency, and natively supports (b) physical interactions, (c) interactive walkthroughs, and (d) ray tracing. (e) MeshSplatting enables straightforward object extraction, allowing scene elements to be directly exported and imported into game engines.
• Figure 2: Mesh parametrization. (left) In a triangle soup, each triangle $\mathbf{T}_m$ is defined independently by three vertices $\mathbf{v}_i$, $\mathbf{v}_j$, $\mathbf{v}_k$, a color $\mathbf{c_m}$, a smoothness parameter $\sigma_m$, and an opacity $o_m$, without sharing vertices with neighboring triangles. (right) MeshSplatting parameterize a triangle $\mathbf{T}_m$ through a shared vertex set, where each vertex $\mathbf{v}_i$ stores $x_i, y_i, z_i$, $c_i$, and $o_i$. Each triangle is defined by the three indices in the vertex set that compose it. During the backward pass, gradients from all adjacent triangles are accumulated at shared vertices. The smoothness parameter $\sigma$ is shared across all triangles.
• Figure 3: From triangle soups to meshes. (1a) We initialize semi-transparent triangles and scale them based on local density. (1b) We optimize a semi-transparent triangle soup without shared vertices, leading to disconnected triangles. (2a) Applying restricted Delaunay triangulation restores global connectivity but introduces geometric artifacts and a loss of visual quality, as vertex colors no longer accurately align with the underlying geometry. (2b) The final fine-tuning stage refines the connected mesh, producing smooth surfaces, accurate geometry, and restoring the visual fidelity lost during triangulation. Using only opaque triangles, our method achieves high visual quality compared to the semi-transparent and isolated triangle soup.
• Figure 4: Window parameter scheduling. To ensure stable gradient flow during training, we begin with smooth triangles ($\sigma{=}1.0$, left) and linearly decrease $\sigma$ throughout training, resulting in sharper triangles by the end. We visualize $\sigma$ for a prototypical triangle at the beginning and end of each optimization stage.
• Figure 5: Qualitative results. Comparison of our method with ground truth, the current state-of-the-art MiLo Guedon2025MILo-arxiv and opaque$\dagger$ Triangle Splatting Held2025Triangle-arxiv. Our approach produces renderings that are closer to the ground truth, with sharper details and finer structures (see the Bicycle spokes), and with fewer artifacts (see the table in the Truck scene). More visualizations are available in the supplementary material.