Material Magic Wand: Material-Aware Grouping of 3D Parts in Untextured Meshes

Abstract

We introduce the problem of material-aware part grouping in untextured meshes. Many real-world shapes, such as scales of pinecones or windows of buildings, contain repeated structures that share the same material but exhibit geometric variations. When assigning materials to such meshes, these repeated parts often require piece-by-piece manual identification and selection, which is tedious and time-consuming. To address this, we propose Material Magic Wand, a tool that allows artists to select part groups based on their estimated material properties -- when one part is selected, our algorithm automatically retrieves all other parts likely to share the same material. The key component of our approach is a part encoder that generates a material-aware embedding for each 3D part, accounting for both local geometry and global context. We train our model with a supervised contrastive loss that brings embeddings of material-consistent parts closer while separating those of different materials; therefore, part grouping can be achieved by retrieving embeddings that are close to the embedding of the selected part. To benchmark this task, we introduce a curated dataset of 100 shapes with 241 part-level queries. We verify the effectiveness of our method through extensive experiments and demonstrate its practical value in an interactive material assignment application.

Figures (16)

• Figure 1: Material Magic Wand. Given an untextured 3D mesh (left) with existing part segmentation, which is often obtained by finding connected components of the mesh, a user can apply our tool to select a group of material-consistent parts by clicking on one single representative part. In the middle, we show example selections of petal, base, sepal, stem, leaf, and grass. For each selection, the tool automatically finds all other parts in the shape that are likely to share the same material (right) through geometric and contextual cues, accelerating the material assignment process. Colors may appear darker due to backface shading.
• Figure 2: Method Overview.Left: Our view selection process renders each part with nearby context from multiple viewpoints sampled randomly over a hemisphere. We choose the one with minimal occlusion, $I^{ctx}$, and use the same viewpoint to render the part in isolation, $I^{part}$. $I^{full}$ captures the entire mesh. We highlight the part with a different color from the rest of the mesh. Right: For each part, its corresponding images are passed through an encoder and their embeddings are concatenated. During training, embeddings of parts with the same material are pulled together, while those with different materials are pushed apart.
• Figure 3: Ambiguities in raw material IDs. Using meshes from Objaverse directly for material grouping can introduce ambiguity in the testing benchmark due to artistic intent, noisy labeling, or coarse material assignment. For example, some shingles on the roof exhibit mixed materials, and fence or wall slats alternate between different material labels (left). To reduce such inconsistencies, we manually refine the material annotations to make the material assignment more uniform and fine-grained (right).
• Figure 4: Precision–Recall curve. We sweep the similarity threshold to evaluate retrieval performance. Our method consistently maintains higher precision across all recall levels, achieving the highest AUC (89.7), followed by DINO (81.1).
• Figure 5: Qualitative comparison. For each mesh, the red part denotes the query, and orange parts indicate the retrieved matches. Our method retrieves components that are both geometrically and contextually similar with the query, while baselines often miss structurally related parts or include visually similar but contextually incorrect ones.