Iskra: A System for Inverse Geometry Processing

TL;DR

Iskra presents an imperative, tensor-based system that enables differentiating through a broad class of geometry processing solvers via implicit differentiation and the adjoint method. By coupling a geometry module with an implicit-function module, it supports sparse linear systems, eigenproblems, local-global iterations, and ADMM, while maintaining tight integration with PyTorch and ML workflows. Its fixed-point and structured implicit layers, along with gather-scatter mesh representations, yield robust performance and memory efficiency across inverse geometry tasks such as mean curvature flow, spectral parameterization, ARAP, and geodesic distances. The work demonstrates practical viability, interoperability with learning pipelines, and significant speedups over declarative baselines, offering a flexible tool for embedding geometric solvers in differentiable programs.

Abstract

We propose a system for differentiating through solutions to geometry processing problems. Our system differentiates a broad class of geometric algorithms, exploiting existing fast problem-specific schemes common to geometry processing, including local-global and ADMM solvers. It is compatible with machine learning frameworks, opening doors to new classes of inverse geometry processing applications. We marry the scatter-gather approach to mesh processing with tensor-based workflows and rely on the adjoint method applied to user-specified imperative code to generate an efficient backward pass behind the scenes. We demonstrate our approach by differentiating through mean curvature flow, spectral conformal parameterization, geodesic distance computation, and as-rigid-as-possible deformation, examining usability and performance on these applications. Our system allows practitioners to differentiate through existing geometry processing algorithms without needing to reformulate them, resulting in low implementation effort, fast runtimes, and lower memory requirements than differentiable optimization tools not tailored to geometry processing.

Figures (9)

• Figure 1: A typical pipeline in iskra: the user maps a parameter vector $\theta$ to mesh quantities $x$, in this case, control handles. Next the user defines an implicit geometric layer, which solves a forward (inner) geometric problem, such as ARAP deformation; iskra automatically provides gradients in the process. Finally, the user uses the solution to the inner problem in an outer problem with loss function $\mathlarger{\ell}$.
• Figure 2: Mesh inflation via inverse mean curvature flow. The Sappho mesh is inflated by treating it as the solution to the mean curvature flow at some time $t$ and optimizing for the initial condition that led to that solution. Larger $t$ results in more inflation.
• Figure 3: Runtime comparison of linear solver strategies across the CPU and GPU. Using a declarative method comes with severe limitation in runtime and memory consumption: Theseus did not succeed at differentiating through systems involving meshes with $>642$ vertices. A rigid system that pre-defines a solver is also insufficient for geometry processing as it is often possible to make use of problem-specific strategies.
• Figure 4: Code comparison for inverse fairing. Top row compares the setup necessary for each framework, and the bottom compares executing one step of fairing.
• Figure 5: Left-to-right: same mesh at three different resolutions of a level-of-detail hierarchy. We optimize a neural metric field on the middle-level mesh to reduce area distortion during SCP. Having optimized once, we transfer the metric to other levels by evaluating the field on other meshes.