Visual Tomography: Physically Faithful Volumetric Models of Partially Translucent Objects

TL;DR

Visual Tomography presents a physically faithful volumetric reconstruction pipeline for partially translucent objects by modeling a heterogeneous interior medium and optimizing absorption, scattering, and light transport with differentiable ray tracing. The method bootstraps a physically-based volume from a non-physical emissive volume (NeRF) and enables relighting, slicing, and immersion into different water-type media, validated on two plankton datasets. Compared with NeRF baselines, the MO+LO configuration yields improved accuracy and novel-view stability, with MO+LO+NH offering the best overall performance by combining density priors with physically-based rendering. The work advances training-data generation and in-situ visualization under varied illumination and media, with potential impact on oceanography, taxonomy, and remote sensing.

Abstract

When created faithfully from real-world data, Digital 3D representations of objects can be useful for human or computer-assisted analysis. Such models can also serve for generating training data for machine learning approaches in settings where data is difficult to obtain or where too few training data exists, e.g. by providing novel views or images in varying conditions. While the vast amount of visual 3D reconstruction approaches focus on non-physical models, textured object surfaces or shapes, in this contribution we propose a volumetric reconstruction approach that obtains a physical model including the interior of partially translucent objects such as plankton or insects. Our technique photographs the object under different poses in front of a bright white light source and computes absorption and scattering per voxel. It can be interpreted as visual tomography that we solve by inverse raytracing. We additionally suggest a method to convert non-physical NeRF media into a physically-based volumetric grid for initialization and illustrate the usefulness of the approach using two real-world plankton validation sets, the lab-scanned models being finally also relighted and virtually submerged in a scenario with augmented medium and illumination conditions. Please visit the project homepage at www.marine.informatik.uni-kiel.de/go/vito

Figures (16)

• Figure 1: Dwarf Prawns MO + LO optimization over time, from up left to right down: iteration 1, 2, 3, 4, 10, 20, 40, 60.
• Figure 2: Praunous Flexuosus MO + LO optimization over time, from up left to right down: iteration 1, 2, 3, 4, 10, 20, 40, 60.
• Figure 3: Image acquisition setup: a macro camera takes brightfield images of small, partially translucent specimens.
• Figure 4: Full image acquisition setup: The specimen is captured in front of an uncalibrated off-the-shelf microscope lightsource, on top of a base, equipped with random dot markers li2013multiple (Left: Dwarf Prawn, Right: Praunus Flexuosus).
• Figure 5: From left to right: optimization result in NeRF, density extracted, as described in Sect. \ref{['sec:nerf_init']} and re-rendered in MO condition with 0 iterations (result shown with 128 spp), refined for 15 iterations in MO+LO+NH condition (result shown with 128spp).