Differentiable Convex Polyhedra Optimization from Multi-view Images

TL;DR

This work tackles differentiable optimization of 3D shapes represented as unions of convex polyhedra by avoiding implicit fields and training solely from image supervision. It blends a non-differentiable duality transform to identify plane intersections with a differentiable solver for the $3$-plane intersection to obtain vertex positions, enabling gradient-based optimization through a differentiable renderer. The approach demonstrates strong performance in shape reconstruction, textured multiview reconstruction, and shape parsing, and includes extensive ablations on densification, spawning, and the number of convexes. By providing a compact, interpretable primitive-based representation with differentiable rendering, the method offers a scalable alternative to implicit-field methods for tasks requiring precise geometry and enabling larger-scale data usage.

Abstract

This paper presents a novel approach for the differentiable rendering of convex polyhedra, addressing the limitations of recent methods that rely on implicit field supervision. Our technique introduces a strategy that combines non-differentiable computation of hyperplane intersection through duality transform with differentiable optimization for vertex positioning with three-plane intersection, enabling gradient-based optimization without the need for 3D implicit fields. This allows for efficient shape representation across a range of applications, from shape parsing to compact mesh reconstruction. This work not only overcomes the challenges of previous approaches but also sets a new standard for representing shapes with convex polyhedra.

Figures (9)

• Figure 1: A novel method optimizes differentiable convex polyhedra w.r.t image losses, bridging the gap between compact shape representation and easily obtained image supervision.
• Figure 2: Overview of the proposed method. Given a set of hyperplanes, we use duality transform to map them into the dual space. We then compute the convex hull of the dual vertices. Each facet of the dual convex hull represent a intersection point in the primal domain. Once the plane IDs for each intersection vertex are recorded, we can recompute the vertex location via differentiable linear equation solvers.
• Figure 3: Small convex polyhedra and redundant planes will be removed to speed up the optimization process. To better reconstruct the shape with high curvature, we employ a densification process that constructs the mesh of the convex polyhedron, runs Loop subdivision, and uses the recomputed convex hull equations of the subdivided mesh to serve as the updated plane parameters.
• Figure 4: Comparing our method with VP, CVXNet, and BSPNet on the ShapeNet dataset. The visualization results show that our method generates better reconstruction, especially in the thin and detailed aspects of shapes.
• Figure 5: We assess our convex polyhedron-based method against DBW monnier2023dbw that is based on boxes and superquadrics. Visual comparisons demonstrate that our approach yields much better reconstruction results.