TeethDreamer: 3D Teeth Reconstruction from Five Intra-oral Photographs

TL;DR

TeethDreamer tackles remote orthodontic monitoring by reconstructing accurate 3D dental models from five intra-oral photographs. It leverages a diffusion-prior to synthesize consistent multi-view color images and normal maps conditioned on segmented teeth, enforces cross-view consistency with a 3D-aware attention mechanism, and performs geometry-aware neural implicit surface reconstruction with a specialized normal loss. The approach achieves superior 3D mesh quality and 2D image realism compared with baselines, validating its potential for remote orthodontic follow-up. While effective, it incurs notable runtime costs and sometimes misses ultra-fine geometric details, motivating future work on speedups and higher-frequency feature integration.

Abstract

Orthodontic treatment usually requires regular face-to-face examinations to monitor dental conditions of the patients. When in-person diagnosis is not feasible, an alternative is to utilize five intra-oral photographs for remote dental monitoring. However, it lacks of 3D information, and how to reconstruct 3D dental models from such sparse view photographs is a challenging problem. In this study, we propose a 3D teeth reconstruction framework, named TeethDreamer, aiming to restore the shape and position of the upper and lower teeth. Given five intra-oral photographs, our approach first leverages a large diffusion model's prior knowledge to generate novel multi-view images with known poses to address sparse inputs and then reconstructs high-quality 3D teeth models by neural surface reconstruction. To ensure the 3D consistency across generated views, we integrate a 3D-aware feature attention mechanism in the reverse diffusion process. Moreover, a geometry-aware normal loss is incorporated into the teeth reconstruction process to enhance geometry accuracy. Extensive experiments demonstrate the superiority of our method over current state-of-the-arts, giving the potential to monitor orthodontic treatment remotely. Our code is available at https://github.com/ShanghaiTech-IMPACT/TeethDreamer

Figures (4)

• Figure 1: This flowchart illustrates our algorithm for reconstructing 3D dental models (c) from multiple intra-oral photographs (a). Initially, we synthesize multi-view images and normal maps (b) using the 3D dental model from our dataset. Subsequently, we train a diffusion network to generate these images and maps directly from the intra-oral photographs, culminating in the reconstruction of the target 3D dental models.
• Figure 2: Overview of TeethDreamer. (a) Generate color images and normal maps at different views from a pretrained diffusion model conditioned by segmented teeth images. Here, the diffusion model denoises the target view $\left\{\mathbf{c}^{i}_t,\mathbf{n}^{i}_t\right\}$ for one step. (b) 3D-aware feature extracted from all target views $\left\{\mathbf{c}^{1:N}_t, \mathbf{n}^{1:N}_t\right\}$ in latent domain to enforce consistency among generated views. (c) Geometry-aware teeth reconstruction from generated color images and normal maps.
• Figure 3: Qualitative comparisons of reconstructed 3D teeth with other baselines, demonstrating our results with complete shapes and geometric details. (GT: ground truth)
• Figure 4: Qualitative comparisons of generated images with other baselines, where our generations are closely aligned with ground truth. (GT: ground truth)