ConceptFactory: Facilitate 3D Object Knowledge Annotation with Object Conceptualization

TL;DR

ConceptFactory, a novel scope to facilitate more efficient annotation of 3D object knowledge by recognizing 3D objects through generalized concepts through generalized concepts, is presented, aiming at promoting machine intelligence to learn comprehensive object knowledge from both vision and robotics aspects.

Abstract

We present ConceptFactory, a novel scope to facilitate more efficient annotation of 3D object knowledge by recognizing 3D objects through generalized concepts (i.e. object conceptualization), aiming at promoting machine intelligence to learn comprehensive object knowledge from both vision and robotics aspects. This idea originates from the findings in human cognition research that the perceptual recognition of objects can be explained as a process of arranging generalized geometric components (e.g. cuboids and cylinders). ConceptFactory consists of two critical parts: i) ConceptFactory Suite, a unified toolbox that adopts Standard Concept Template Library (STL-C) to drive a web-based platform for object conceptualization, and ii) ConceptFactory Asset, a large collection of conceptualized objects acquired using ConceptFactory suite. Our approach enables researchers to effortlessly acquire or customize extensive varieties of object knowledge to comprehensively study different object understanding tasks. We validate our idea on a wide range of benchmark tasks from both vision and robotics aspects with state-of-the-art algorithms, demonstrating the high quality and versatility of annotations provided by our approach. Our website is available at https://apeirony.github.io/ConceptFactory.

Figures (7)

• Figure 1: [Left] Illustration of the relationship between human cognition (a-b) and our approach (c-e), exemplified by handle as object and affordable interaction as knowledge. (a) Human recognizes objects as an arrangement of geometric components. (b) Abstract commonsense information are induced from the geometries in human mind. (c) Explicitly model the abstract information as a regular geometry concept with specific knowledge. (d) Generalize the concept towards different objects. (e) Propagate the knowledge from the concept to objects as annotations. [Right] Example of parameters and the constructor of a concept template. Please refer to the codes in our website for concept template implementations.
• Figure 2: Shape instances of geometry (Top) and concept (Bottom) templates with specific parameters. [Bottom] The figures on the left side of the arrows display each geometry component of a concept template individually, whereas those on the right side are example instances of concept templates with various parameters. The instance at bottom-right is the result of modifying discrete parameters.
• Figure 3: [Left] Minor gaps in geometric details between the original object (bottom) and its conceptualization (top). [Right] Restoring the geometric details via deformation based on point-wise correspondences.
• Figure 4: An overview to our conceptualization interface and the workflow (blue arrow). The interface is divided into four components: work space, target view, template rendering, and mixed view. In work space, users first select best-match templates for each part of the target object, then parameterize each concept template with the help of the optimizer, and finally save the conceptualization result. Target view illustrates the shape of the target object, template rendering displays instances of concept templates with current parameters, while mixed view visualizes the integration between target view (gray) and template rendering (blue), helping users perform the conceptualization efficiently. Zoom in for a clear view.
• Figure 5: Examples of differentiable deformations on template instances with default parameters. [Left] A default quadrangular prism instance in mesh with all edge lengths set to 1, bearing the same shape as a cuboid. Its eight vertices are labelled from $v_1$ to $v_8$. The upper case show a translation is applied to four top vertices for a 3d parallelogram, and the lower case show a scaling is applied to four bottom vertices for a frustum of a pyramid. [Right] A sphere mesh with radius one and set of vertices $\mathbf{V}$. The upper case show a scaling is applied to $y$-axis of all vertices for an ellipsoid, and the lower case show we use ReLU operation to truncate a sphere. Through differentiable transformations (addition, multiplication, etc.) on their respective vertices, the shapes can be deformed in a differentiable manner.