PReP: Efficient context-based shape retrieval for missing parts

TL;DR

Part Retrieval Pipeline is presented, a pipeline that creatively utilizes metric learning techniques along with a trained classification model to measure the suitability of potential replacement parts from a database, as part of an application scenario targeting circular economy.

Abstract

In this paper we study the problem of shape part retrieval in the point cloud domain. Shape retrieval methods in the literature rely on the presence of an existing query object, but what if the part we are looking for is not available? We present Part Retrieval Pipeline (PReP), a pipeline that creatively utilizes metric learning techniques along with a trained classification model to measure the suitability of potential replacement parts from a database, as part of an application scenario targeting circular economy. Through an innovative training procedure with increasing difficulty, it is able to learn to recognize suitable parts relying only on shape context. Thanks to its low parameter size and computational requirements, it can be used to sort through a warehouse of potentially tens of thousand of spare parts in just a few seconds. We also establish an alternative baseline approach to compare against, and extensively document the unique challenges associated with this task, as well as identify the design choices to solve them.

Figures (10)

• Figure 1: An overview of the proposed methods. An query object is missing a part and we need to find a replacement among a given database of parts. Using our proposed method, PReP, the matching part can be found only based on the context. By adding an additional completion step, standard shape retrieval techniques can be applied to retrieved a similar part.
• Figure 2: This figure presents an overview of our proposed pipeline. The input shape (in this case, a toy plane) is split into its individual parts manually or through a segmentation network and each part is encoded into a feature vector. The feature vectors are then fed into the transformer to model the relationships between them. At this stage the pre-encoded spare parts are also introduced to the transformer one at a time. For each spare part, the object is assessed as a whole, and a suitability score is provided by the trained classifier. (Each module is labeled with the corresponding mathematical symbol which is used to describe it in the text)
• Figure 3: Visualization of $g$ for varying levels of steepness. The steeper the curve, the bigger the transition is from "highly similar" to "dissimilar" between objects. By gradually increasing the difficulty of the task the model becomes better at assigning feature points for parts based on their similarity.
• Figure 4: Retrieval results on the Partnet dataset. The same query table can be seen in two scenarios; one with a missing leg and one with a missing surface. The shapes on the top and bottom rows (color coded with green and red) represent the top and worst 5 matches respectively. Despite the various imperfections deriving from improper clustering the model is able to pick out fitting replacements. The colormap represents the normalized z-coordinate of each point and assists in understanding the object's geometry.
• Figure 5: Retrieval results on the ShapeNet-part dataset. The same query plane can be seen in two scenarios; with missing wings and with missing fuselage. The parts on the top and bottom rows represent the best and worst matches respectively. The model is able to achieve good retrieval quality, despite the data being undersampled and the parts often being mislabeled.