Mesh-Informed Neural Operator : A Transformer Generative Approach

TL;DR

The paper tackles the challenge of generating functions in infinite-dimensional spaces on irregular domains by introducing the Mesh-Informed Neural Operator (MINO), a domain- and discretization-agnostic backbone that combines a graph neural operator based geometry encoder with cross-attention between encoder and decoder. It can operate with an optional latent processor (eg, a diffusion U‑Net) to enhance generative performance and uses flow matching to learn velocity fields $v_t$ that transport a base measure $mu_0$ to a target measure $mu_1$. To enable fair comparison across datasets and discretizations, the paper proposes standardized metrics $SWD$ and $MMD$ for functional generative models and demonstrates state-of-the-art performance on six benchmarks, both on regular and irregular grids, with substantial efficiency gains over strong baselines. By unifying neural operators with modern DL architectures, MINO broadens the applicability of high-fidelity function-space generation to complex scientific problems and irregular geometries, and the accompanying code facilitates broader adoption.

Abstract

Generative models in function spaces, situated at the intersection of generative modeling and operator learning, are attracting increasing attention due to their immense potential in diverse scientific and engineering applications. While functional generative models are theoretically domain- and discretization-agnostic, current implementations heavily rely on the Fourier Neural Operator (FNO), limiting their applicability to regular grids and rectangular domains. To overcome these critical limitations, we introduce the Mesh-Informed Neural Operator (MINO). By leveraging graph neural operators and cross-attention mechanisms, MINO offers a principled, domain- and discretization-agnostic backbone for generative modeling in function spaces. This advancement significantly expands the scope of such models to more diverse applications in generative, inverse, and regression tasks. Furthermore, MINO provides a unified perspective on integrating neural operators with general advanced deep learning architectures. Finally, we introduce a suite of standardized evaluation metrics that enable objective comparison of functional generative models, addressing another critical gap in the field.

Figures (8)

• Figure 1: Overview of the MINO architecture. The geometry encoder uses a GNO as a domain-agnostic tokenizer, followed by several cross-attention blocks and an optional latent processor. The decoder then employs a distinct cross-attention mechanism to map the latent representation back to the target locations.
• Figure 2: Inference and zero-shot Generation with MINO. (a) MINO gradually transforms a GP sample to the data sample under flow matching paradigm. (b) Zero-shot generation at varying spatial scales by directly transforming finer GP samples to finer data samples.
• Figure 3: Visualization of generation and zero-shot super-resolution by MINO-U. (a) Navier-Stokes samples generated on the original mesh (4,096 nodes) and a finer mesh (25,600 nodes). (b) Global Climate sample generated on the original mesh (4,140 nodes) and a finer mesh (16,560 nodes).
• Figure 4: Consistency of SWD and MMD with respect to the number of discretization points. The metrics remain stable as the number of observation points is varied for three comparison scenarios: two synthetic GPs (left), a GP vs. a regular grid dataset (center), and a GP vs. an irregular mesh dataset (right).
• Figure 5: Generation of Navier-Stokes samples under different baselines