LeafFit: Plant Assets Creation from 3D Gaussian Splatting

TL;DR

LeafFit addresses the gap between high-fidelity 3D Gaussian Splatting reconstructions of plants and game-ready mesh assets by exploiting leaf repetition. It segments leaves from Gaussians, selects a representative template leaf, and applies differentiable Moving Least Squares deformation $\Phi_j$ to fit every leaf to the template, with on-the-fly mesh deformation in a vertex shader and compact per-leaf parameters $C_j$. A template leaf mesh is extracted via Ball Pivoting (BPA) with texture transfer, enabling real-time rendering as $O(|V|)$ shared memory plus $O(KN)$ per plant, where $K$ is per-leaf control points and $N$ the leaf count, and MLS runs in $O(|V|K)$. Experiments show higher segmentation accuracy and deformation fidelity than baselines, along with substantial data compression and seamless integration into standard asset pipelines for parameter-level editing.

Abstract

We propose LeafFit, a pipeline that converts 3D Gaussian Splatting (3DGS) of individual plants into editable, instanced mesh assets. While 3DGS faithfully captures complex foliage, its high memory footprint and lack of mesh topology make it incompatible with traditional game production workflows. We address this by leveraging the repetition of leaf shapes; our method segments leaves from the unstructured 3DGS, with optional user interaction included as a fallback. A representative leaf group is selected and converted into a thin, sharp mesh to serve as a template; this template is then fitted to all other leaves via differentiable Moving Least Squares (MLS) deformation. At runtime, the deformation is evaluated efficiently on-the-fly using a vertex shader to minimize storage requirements. Experiments demonstrate that LeafFit achieves higher segmentation quality and deformation accuracy than recent baselines while significantly reducing data size and enabling parameter-level editing.

Figures (12)

• Figure 1: No two leaves belongs to a plant are exactly the same, but they are similar. We compress the data by replacing the leaves by deformed template leaf.
• Figure 2: Workflow. (a) Input: a pre-trained set of Gaussian primitives $\mathcal{G}$. (b) The user selects a root primitive $r$; we compute geodesic distances from $r$, detect apexes $a$ as local maxima, and build a tree. The tree graph is used to find leaf petiole. (c) Using petiole and apex cues, we segment leaves and let the user choose a template leaf $\mathcal{L}_t$. (d) Other leaves are fitted to $\mathcal{L}_t$ via Moving Least Squares deformation. (e) Output: an efficient, editable instanced mesh suitable for game assets.
• Figure 3: Tree-graph construction. (a) Start from the full set of Gaussians $\mathcal{G}$. (b) Downsample to a sparse subset $\bar{\mathcal{G}}$ to reduce the cost of detecting apexes via local maxima on the geodesic field (from the root $r$). (c) For each apex $\alpha$, trace a rootward path by greedy descent on geodesic distance; when it meets an existing path, attach it as a branch to complete the tree.
• Figure 4: Separating apex pairs via triangle inequality. Given two apexes $a$ and $a'$ with lowest common ancestor $\mathrm{lca}(a,a')$ on the rootward tree, let $D_a[\cdot]$ and $D_{a'}[\cdot]$ denote geodesic distances from $a$ and $a'$, respectively. Left: Apexes on different leaves satisfy $D_a[a'] \approx D_a[\mathrm{lca}(a,a')] + D_{a'}[\mathrm{lca}(a,a')]$. Right: Apexes on the same leaf yield a strictly shorter direct path, $D_a[a'] < D_a[\mathrm{lca}(a,a')] + D_{a'}[\mathrm{lca}(a,a')] - \tau$. We use this criterion to group apexes into leaves.
• Figure 5: Leaf base determination. For each apex $a$, we compute the geodesic distances $D_a[\cdot]$ and traverse the rootward path $\mathcal{P}_a$. At each path point $p_k$, we query the set of primitives $\mathcal{G}_{p_k}$ within an iso-geodesic band of width $\delta$ to compute the local diameter $d_{p_k}$. The traversal terminates when $d_{p_k}$ drops below the threshold $\epsilon$, marking the petiole.