Distill3R: A Pipeline for Democratizing 3D Foundation Models on Commodity Hardware

TL;DR

Distill3R tackles the compute barrier in 3D foundation-model research by introducing offline teacher caching and a confidence-aware distillation loss to train a compact $72~\mathrm{M}$ student on a single workstation. The method yields a $9\times$ parameter reduction and a $5\times$ inference speedup compared to a $650~\mathrm{M}$ teacher, with training completed in under 3 days on commodity GPUs. The student preserves structural and topological 3D understanding while achieving better scale fidelity (near-metric scale) across indoor and OOD scenes, making it suitable for edge deployment and robotics. By providing a reproducible, end-to-end training recipe, Distill3R offers a practical, accessible platform for labs lacking large-scale infrastructure to explore domain-specific 3D vision through distillation-based baselines and rapid iteration.

Abstract

While multi-view 3D reconstruction has shifted toward large-scale foundation models capable of inferring globally consistent geometry, their reliance on massive computational clusters for training has created a significant barrier to entry for most academic laboratories. To bridge this compute divide, we introduce Distill3R, a framework designed to distill the geometric reasoning of 3D foundation models into compact students fully trainable on a single workstation. Our methodology centers on two primary innovations: (1) an offline caching pipeline that decouples heavy teacher inference from the training loop through compressed supervision signals, and (2) a confidence-aware distillation loss that leverages teacher uncertainty to enable training on commodity hardware. We propose a 72M-parameter student model which achieves a 9x reduction in parameters and a 5x inference speedup compared to its 650M-parameter teacher. The student is fully trainable in under 3 days on a single workstation, whereas its teacher requires massive GPU clusters for up to a week. We demonstrate that the student preserves the structural consistency and qualitative geometric understanding required for functional 3D awareness. By providing a reproducible, single-workstation training recipe, Distill3R serves as an exploratory entry point for democratized 3D vision research and efficient edge deployment. This work is not intended to compete with state-of-the-art foundation models, but to provide an accessible research baseline for laboratories without access to large-scale compute to train and specialize models on their own domain-specific data at minimal cost.

Figures (5)

• Figure 1: An overview of the offline teacher caching pipeline. We decouple training by pre-computing teacher supervision. A Manifest Generation module indexes raw data, which is processed by Fast3R in fixed-size samples. The caching module compresses predictions (float16) and masks (RLE) into a compressed static cache used by the student and teacher dataset modules for training.
• Figure 2: Student Architecture. DUNE ViT-S encoder followed by a 6L/384d Global Fusion Transformer and DPT heads.
• Figure 3: An overview of the Distill3R training loop. (1) The Distill3R Student Dataset Module loads a batch of $N$ images (from the Image Manifest) and the corresponding $N$ pre-computed supervision signals (from the Teacher Cache). (2) The Student Model processes the images and generates its own 3D point maps and confidence maps. (3) The Distill3R Loss Module compares the Student Outputs against the cached Teacher Outputs to compute $\mathcal{L}_{\text{total}}$. (4) The resulting Gradient Flow (red dotted line) updates only the Student Model's parameters.
• Figure 4: Qualitative comparison on 7-Scenes (OOD). Side-by-side reconstructions showing that Distill3R (right) preserves the global topology of the teacher (left).
• Figure 5: Object reconstruction (CO3D-v2). Qualitative results for the Distill3R student on object-centric scenes.