PolyNODE: Variable-dimension Neural ODEs on M-polyfolds

TL;DR

It is demonstrated experimentally that the PolyNODE models can be trained to solve reconstruction tasks in M-polyfolds and that latent representations of the input can be extracted and used to solve downstream classification tasks.

Abstract

Neural ordinary differential equations (NODEs) are geometric deep learning models based on dynamical systems and flows generated by vector fields on manifolds. Despite numerous successful applications, particularly within the flow matching paradigm, all existing NODE models are fundamentally constrained to fixed-dimensional dynamics by the intrinsic nature of the manifold's dimension. In this paper, we extend NODEs to M-polyfolds (spaces that can simultaneously accommodate varying dimensions and a notion of differentiability) and introduce PolyNODEs, the first variable-dimensional flow-based model in geometric deep learning. As an example application, we construct explicit M-polyfolds featuring dimensional bottlenecks and PolyNODE autoencoders based on parametrised vector fields that traverse these bottlenecks. We demonstrate experimentally that our PolyNODE models can be trained to solve reconstruction tasks in these spaces, and that latent representations of the input can be extracted and used to solve downstream classification tasks. The code used in our experiments is publicly available at https://github.com/turbotage/PolyNODE .

Figures (10)

• Figure 1: Illustration (grey shading) of the stratified topological space $\Omega^1_1 \subset \mathop{\mathrm{\mathbb{R}}}\nolimits^3$ with $\tau_1=-2$ and $\tau_2=2$.
• Figure 2: Illustration of a semi-flow in $\Omega^1_1 \subset \mathop{\mathrm{\mathbb{R}}}\nolimits^3$ with $\tau_1=-2$ and $\tau_2=2$, polyfold structure indicated as grey planes.
• Figure 3: Illustration of a flow line entering the bottleneck at $\tau_1$ and exiting at $\tau_2$. Explicit compression starts at $\tau_0$.
• Figure 4: (a) Flow lines (green) for individual input samples for $N=1$. Input set $\mathcal{X}$ (blue) and reconstructed output $\hat{\mathcal{Y}}$ (orange). The $y_2$ component is projected out for visualisation. (b) Time slices of the flow for a spiral with $N=4$. Colour scale corresponds to the angular parameter.
• Figure 5: Relative monotonicity error during training for $N=4$.