GARF: Learning Generalizable 3D Reassembly for Real-World Fractures

TL;DR

This work addresses the challenge of generalizing 3D fracture reassembly from synthetic data to real-world fractures. It introduces GARF, a flow-based SE(3) framework augmented with fracture-aware pretraining, two-session inference, multi-anchor training, and LoRA fine-tuning, and validates it on Fractura, a diverse real/synthetic fracture dataset. GARF achieves state-of-the-art performance across synthetic and real-world scenarios, notably reducing rotation error and boosting part accuracy, while demonstrating robustness to missing or extraneous fragments. The Fractura dataset and GARF’s generalization capabilities offer a new benchmark and methodology for cross-domain 3D puzzle solving in archaeology, paleontology, and related fields.

Abstract

3D reassembly is a challenging spatial intelligence task with broad applications across scientific domains. While large-scale synthetic datasets have fueled promising learning-based approaches, their generalizability to different domains is limited. Critically, it remains uncertain whether models trained on synthetic datasets can generalize to real-world fractures where breakage patterns are more complex. To bridge this gap, we propose GARF, a generalizable 3D reassembly framework for real-world fractures. GARF leverages fracture-aware pretraining to learn fracture features from individual fragments, with flow matching enabling precise 6-DoF alignments. At inference time, we introduce one-step preassembly, improving robustness to unseen objects and varying numbers of fractures. In collaboration with archaeologists, paleoanthropologists, and ornithologists, we curate Fractura, a diverse dataset for vision and learning communities, featuring real-world fracture types across ceramics, bones, eggshells, and lithics. Comprehensive experiments have shown our approach consistently outperforms state-of-the-art methods on both synthetic and real-world datasets, achieving 82.87\% lower rotation error and 25.15\% higher part accuracy. This sheds light on training on synthetic data to advance real-world 3D puzzle solving, demonstrating its strong generalization across unseen object shapes and diverse fracture types. GARF's code, data and demo are available at https://ai4ce.github.io/GARF/.

Figures (12)

• Figure 1: We curate Fractura, a unique dataset presenting real-world fracture assembly challenges across scientific domains, including ceramics, bones, eggshells, and lithics. To tackle these challenges, we introduce Garf, a generalizable 3D reassembly framework designed to handle varying object shapes, diverse fracture types, and the presence of missing or extraneous fragments.
• Figure 2: Characteristics and Challenges of Fractura. (i) Diverse fracture types including two synthetic and three real-world types, across ceramics, bones, eggshells, and lithics. (ii) Real-world challenges such as missing or extraneous fragments.
• Figure 3: Pipeline of Garf. Our framework comprises two main components: (i) Fracture-aware pretraining leverages $14\times$ more data than previous methods to learn the local fracture features via fracture point segmentation, and (ii) Two-session flow-based reassembly on $\mathrm{SE}(3)$ leverages the $\mathrm{SO}(3)$ manifold for precise rotation estimation. At inference time, one-step pre-assembly strategy provides better initial poses, enhancing robustness against unseen objects and increasing numbers of fractures.
• Figure 4: Qualitative Comparisons on the Breaking Bad and Fractura.Garf consistently produces more accurate reassemblies, particularly on the Breaking Bad Artifact subset and Fractura synthetic fracture subset, demonstrating strong generalization to unseen object shapes. Meshes are used for visualization only. Additional results are available in the supplementary material.
• Figure 5: Qualitative Comparisons on the Fractura real fracture subset.Garf generalizes well to random breakage (limb bones and ceramics) and incomplete ossification (vertebrae) but faces challenges with high-ambiguity fractures like flintknapping (lithics). Fine-tuning enhances performance, particularly for thin-shell structures (eggshells) and flintnapping (lithics).