SOFTooth: Semantics-Enhanced Order-Aware Fusion for Tooth Instance Segmentation

TL;DR

SOFTooth tackles the challenging task of 3D tooth instance segmentation in crowded arches, missing teeth, and rare third molars by integrating rich 2D semantics from a frozen SAM model into a 3D dental geometry backbone. It introduces three synergistic components: PRG, which injects occlusal-view SAM embeddings into 3D features to sharpen boundaries; CMR, which enforces center--mask geometric consistency and stabilizes instance centers; and FHM, which augments matching with anatomical FDI order and center-distance priors for coherent labeling. Across 3DTeethSeg'22, SOFTooth achieves state-of-the-art OA and mIoU, with pronounced gains for third molars and undercrowded or partially erupted dentitions, all without 2D mask supervision. The framework is practical for clinical deployment due to its single occlusal-view SAM pass, compatibility with standard 3D backbones, and end-to-end training, enabling more reliable automated dental diagnostics and planning.

Abstract

Three-dimensional (3D) tooth instance segmentation remains challenging due to crowded arches, ambiguous tooth-gingiva boundaries, missing teeth, and rare yet clinically important third molars. Native 3D methods relying on geometric cues often suffer from boundary leakage, center drift, and inconsistent tooth identities, especially for minority classes and complex anatomies. Meanwhile, 2D foundation models such as the Segment Anything Model (SAM) provide strong boundary-aware semantics, but directly applying them in 3D is impractical in clinical workflows. To address these issues, we propose SOFTooth, a semantics-enhanced, order-aware 2D-3D fusion framework that leverages frozen 2D semantics without explicit 2D mask supervision. First, a point-wise residual gating module injects occlusal-view SAM embeddings into 3D point features to refine tooth-gingiva and inter-tooth boundaries. Second, a center-guided mask refinement regularizes consistency between instance masks and geometric centroids, reducing center drift. Furthermore, an order-aware Hungarian matching strategy integrates anatomical tooth order and center distance into similarity-based assignment, ensuring coherent labeling even under missing or crowded dentitions. On 3DTeethSeg'22, SOFTooth achieves state-of-the-art overall accuracy and mean IoU, with clear gains on cases involving third molars, demonstrating that rich 2D semantics can be effectively transferred to 3D tooth instance segmentation without 2D fine-tuning.

Figures (5)

• Figure 1: Existing limitations vs. our improvements. (a) The benchmark dataset 3DTeethSeg'22 shows an imbalanced class distribution, with third molars appearing in only about 5% of cases. (b) Native 3D baselines predominantly rely on 3D geometric features (e.g., coordinates, normals) without effective semantic guidance or structural constraints, failing to address the spatial complexity of crowded 16-tooth cases. (c) Our method exploits 2D--3D semantic fusion for boundary refinement, center-mask consistency to stabilize centers and improve mask reliability, and order-aware assignment to maintain anatomical tooth order and reduce mislabeling of third molars.
• Figure 2: Architecture of SOFTooth for 3D tooth instance segmentation. A dual-stream 3D backbone extracts geometric features from intraoral meshes. The Point-wise Residual Gating (PRG) module renders an occlusal view, samples frozen SAM embeddings via geometry-aligned projection, and injects them into the 3D features through center-guided, point-wise residual gating, yielding boundary-sensitive fused representations that sharpen inter-tooth and tooth-gingiva boundaries. The Center-Guided Mask Refinement (CMR) module learns instance queries and dynamic kernels, and aligns masks with 3D geometric centers to correct center drift and refine tooth instances. The FDI Order-Aware Hungarian Matching (FHM) module performs order-aware matching in similarity space with integrated center distance, yielding anatomically coherent tooth indices for supervision.
• Figure 3: The structure of Point-wise Residual Gating for 2D--3D Fusion (PRG) module. Face projections on the occlusal view are used to bilinearly sample per-face SAM features and center-guided Gaussian weights, which serve as 2D priors on the mesh. A point-wise gating block then injects these priors into 3D face features via a residual connection, sharpening tooth-gingiva and inter-tooth boundaries.
• Figure 4: Qualitative results compared with state-of-the-art methods. (a) 14-Teeth (wisdom teeth excluded), (b) 16-Teeth (complete dentition; normal/crowded), (c) 15-Teeth (single third molar missing—left/right), and (d) complex partial dentitions ($\ge$2 missing teeth). Red dotted circles indicate segmentation distinctions of recent state-of-the-art methods, and the proposed method maintains better segmentation performance and robustness.
• Figure 5: Qualitative ablation of the proposed modules on 3D tooth instance segmentation (corresponding to settings No. 1--No. 4 in Table \ref{['tab:ABLATION']}). Red dotted circles highlight regions where the baseline and our PRG/CMR/FHM variants differ in tooth-gingiva leakage, inter-tooth boundary delineation, center drift, and FDI label consistency. (a) Baseline (No. 1) vs. Baseline + PRG (No. 2). (b) Baseline (No. 1) vs. Baseline + CMR (No. 3). (c) Baseline (No. 1) vs. Baseline + FHM (No. 4).