FUSE: A Flow-based Mapping Between Shapes

TL;DR

FUSE presents a flow-matching framework that represents maps between 3D shapes as the composition of invertible flows between embedding distributions and a shared Gaussian anchor. By training two independent flows to map from the anchor to each shape’s embeddings, FUSE achieves zero-shot, bijective correspondences across meshes, point clouds, SDFs, and volumes without per-pair optimization. The approach delivers high coverage and accuracy on standard non-rigid matching benchmarks and extends naturally to inter-representation tasks and downstream applications such as UV parametrization, skinning, and volume matching. Its probabilistic, representation-agnostic nature enables flexible utilization as a backbone for diverse geometric tasks and prompts further exploration of flow-based alignment in 3D geometry.

Abstract

We introduce a novel neural representation for maps between 3D shapes based on flow-matching models, which is computationally efficient and supports cross-representation shape matching without large-scale training or data-driven procedures. 3D shapes are represented as the probability distribution induced by a continuous and invertible flow mapping from a fixed anchor distribution. Given a source and a target shape, the composition of the inverse flow (source to anchor) with the forward flow (anchor to target), we continuously map points between the two surfaces. By encoding the shapes with a pointwise task-tailored embedding, this construction provides an invertible and modality-agnostic representation of maps between shapes across point clouds, meshes, signed distance fields (SDFs), and volumetric data. The resulting representation consistently achieves high coverage and accuracy across diverse benchmarks and challenging settings in shape matching. Beyond shape matching, our framework shows promising results in other tasks, including UV mapping and registration of raw point cloud scans of human bodies.

Figures (9)

• Figure 1: Our representation establishes a map between shapes $S_1,S_2$ by optimizing invertible flows (red box) between embedding distributions (blue box) and the unit Gaussian $\mathcal{N}(0,1)$. The composition of said flows (green box) is a continuous pointwise map between the surfaces, enabling shape matching and supporting various 3D representations, including neural signed distance fields (orange box).
• Figure 2: Visual example of the proposed pipeline. For visualization purpose, we select a pair for which we can solve for the correspondence using the 3D coordinates as features embedding. Estimated correspondence is encoded by the color (matched points have the same color).
• Figure 3: Effect of FUSE visualized on the first three coordinates of embedding and on the relative obtained map.
• Figure 4: Correspondences obtained by FUSE with different embeddings. 3D coordinates $(x,y,z)$ do not encode the semantic correspondence and cannot capture the non-rigid deformation. WKS does not solve intrinsic symmetries of the shape. With leaned embeddings (FMNet), our methods introduces small artifacts.
• Figure 5: Qualitative evaluation: We report a pair of shapes with visualization of the p2p as transfered colormap and in terms of the geodesic error visualized via a hot-colormap. We note the lower but less smooth error of Ours compared to FMaps.