TopoGaussian: Inferring Internal Topology Structures from Visual Clues

TL;DR

TopoGaussian addresses inferring interior topology of opaque objects from exterior visual cues by combining Gaussian Splatting for exterior reconstruction with a volumetric, mesh-free point cloud and a differentiable particle-based simulator. It supports three topology representations (point, neural implicit surface, and quadratic surface) and optimizes topology parameters via gradients to match observed motion, enabling physically plausible interior designs. The approach yields substantial speedups over mesh-based baselines and improves topology quality, with successful validation on synthetic data and four real-world prototypes including 3D-printed parts. This mesh-free pipeline has potential impact on 3D vision, soft robotics, and manufacturing by enabling non-intrusive interior design inference and rapid fabrication.

Abstract

We present TopoGaussian, a holistic, particle-based pipeline for inferring the interior structure of an opaque object from easily accessible photos and videos as input. Traditional mesh-based approaches require tedious and error-prone mesh filling and fixing process, while typically output rough boundary surface. Our pipeline combines Gaussian Splatting with a novel, versatile particle-based differentiable simulator that simultaneously accommodates constitutive model, actuator, and collision, without interference with mesh. Based on the gradients from this simulator, we provide flexible choice of topology representation for optimization, including particle, neural implicit surface, and quadratic surface. The resultant pipeline takes easily accessible photos and videos as input and outputs the topology that matches the physical characteristics of the input. We demonstrate the efficacy of our pipeline on a synthetic dataset and four real-world tasks with 3D-printed prototypes. Compared with existing mesh-based method, our pipeline is 5.26x faster on average with improved shape quality. These results highlight the potential of our pipeline in 3D vision, soft robotics, and manufacturing applications.

Figures (27)

• Figure 1: Pipeline overview (Sec. \ref{['sec:overview']}). Our pipeline takes as input multi-view photos of an opaque object and its video of motion. We run Gaussian splatting on the multi-view images to obtain a point cloud characterizing its surface geometry and appearance. Next, we refine and fill in internal points to obtain a volumetric point cloud and use our topology representation (with three flexible choices) to attach physical parameters on it. We then simulate the volumetric point cloud with our particle-based differentiable simulator, which compares its simulated motion with that in a reference image or video and backpropagates the gradient of the motion difference to the topology representation. Finally, we perform the optimization algorithm based on the gradient from the simulator and obtain the resulting structure that matches the input motion.
• Figure 2: Six rigid experiments (Sec. \ref{['sec:result:main']}). For each experiment, left: initial balancing position of the object with fully solid topology structure; middle: final balancing position of the object with the result from our pipeline; right: optimization target.
• Figure 3: Comparison on optimization loss (left) and preprocessing time (right). "Fail" on the top of the bar means that the method fails to output visually correct result in the example.
• Figure 4: Comparison between our method and mesh baselines (Sec. \ref{['sec:result:syn']}). Red part represents solid part and blue part represents hollow part. Top row: our method; bottom three rows: mesh baselines.
• Figure 5: One soft body expereiment (Sec. \ref{['sec:result:main']}). Left part: Top row: frames from simulated motion of a fully soft initial guess; middle row: simulated motion of our optimized result; bottom row: target motion. The red rectangles mark the major different parts. Right part: slice of the optimized topology structure with red representing soft part and blue representing hard part.