ShapeICP: Iterative Category-level Object Pose and Shape Estimation from Depth

TL;DR

ShapeICP tackles category-level object pose and shape estimation from a single depth image without pose-annotated training data by introducing a mesh-based Active Shape Model (ASM) and an ICP-inspired alternating optimization that jointly estimates the pose $(R,t,s)$ and shape code $\boldsymbol{c}$. The method integrates EM-based soft correspondences, multi-hypothesis rotation handling, depth-rendering constraints, and symmetry checks to improve robustness in incomplete depth observations. On the NOCS REAL benchmark, ShapeICP is competitive with learning-based approaches and can achieve high accuracy when it converges, demonstrating the viability of a learning-free, geometry-driven approach for category-level estimation. The work highlights the potential for deploying pose/shape estimation in new environments without data curation and suggests future directions including RGB-informed constraints and improved initialization to broaden the convergence basin.

Abstract

Category-level object pose and shape estimation from a single depth image has recently drawn research attention due to its potential utility for tasks such as robotics manipulation. The task is particularly challenging because the three unknowns, object pose, object shape, and model-to-measurement correspondences, are compounded together, but only a single view of depth measurements is provided. Most of the prior work heavily relies on data-driven approaches to obtain solutions to at least one of the unknowns, and typically two, risking generalization failures if not designed and trained carefully. The shape representations used in the prior work also mainly focus on point clouds and signed distance fields (SDFs). In stark contrast to the prior work, we approach the problem using an iterative estimation method that does not require learning from pose-annotated data. Moreover, we construct and adopt a novel mesh-based object active shape model (ASM), which additionally maintains vertex connectivity compared to the commonly used point-based object ASM. Our algorithm, ShapeICP, is based on the iterative closest point (ICP) algorithm but is equipped with additional features for the category-level pose and shape estimation task. Although not using pose-annotated data, ShapeICP surpasses many data-driven approaches that rely on pose data for training, opening up a new solution space for researchers to consider.

Figures (6)

• Figure 1: An example of the template deformation process.
• Figure 2: Visualization of the vertices of the bases and the final shape in an example ASM. The final shape on the right shows how the mean shape is displaced by the weighted bases (the scale of the bases is much smaller than the mean, so the bases on the left are enlarged for clarity).
• Figure 3: The back-projected depths (blue points) are fragmented due to a partial view and occlusion, which causes ambiguity such that the models (red) fit the measurements well but have pose errors (in collision with each other).
• Figure 4: Visualization of example NOCS REAL wang2019normalized frames. From left to right are the color image, the color image overlaid by the depth projection of the estimated objects, the back-projected measured depth image, and the estimated objects from a novel viewpoint.
• Figure 5: Visualization of the original hard one-to-one association (left) and the EM soft associations for $Q=3$ (right). $\mathcal{M}(\mathbf{c})$ is the ASM model that returns a mesh. The blue curve represents the mesh surface transformed by the rotation $R$, translation $t$, and scale $s$. The black dotted curve is the surface of the measured object.