Smoothness Errors in Dynamics Models and How to Avoid Them
Edward Berman, Luisa Li, Jung Yeon Park, Robin Walters
TL;DR
This work analyzes smoothness errors in dynamics models that use graph and mesh neural networks, showing that strictly unitary convolutions can be overly constraining for diffusion-like PDE tasks. It derives a theoretical lower bound on unitary approximation error and introduces relaxed unitary convolutions, including Taylor truncation and encoder-decoder strategies, to balance smoothness preservation with expressive power. The framework is extended from graphs to meshes via mesh Rayleigh quotients and unitary mesh convolutions, enabling PDE surrogates on complex geometries. Empirical results on heat and wave PDEs on PyVista meshes and WeatherBench WB2 demonstrate that relaxed-unitary models (R-UniGraph, R-UniMesh) achieve state-of-the-art or competitive performance, particularly excelling in diffusive dynamics by aligning the learned smoothness with the true system’s smoothness. The findings highlight the importance of controllable smoothness in physics-guided learning and offer practical tools for accurate, mesh-aware PDE modeling.
Abstract
Modern neural networks have shown promise for solving partial differential equations over surfaces, often by discretizing the surface as a mesh and learning with a mesh-aware graph neural network. However, graph neural networks suffer from oversmoothing, where a node's features become increasingly similar to those of its neighbors. Unitary graph convolutions, which are mathematically constrained to preserve smoothness, have been proposed to address this issue. Despite this, in many physical systems, such as diffusion processes, smoothness naturally increases and unitarity may be overconstraining. In this paper, we systematically study the smoothing effects of different GNNs for dynamics modeling and prove that unitary convolutions hurt performance for such tasks. We propose relaxed unitary convolutions that balance smoothness preservation with the natural smoothing required for physical systems. We also generalize unitary and relaxed unitary convolutions from graphs to meshes. In experiments on PDEs such as the heat and wave equations over complex meshes and on weather forecasting, we find that our method outperforms several strong baselines, including mesh-aware transformers and equivariant neural networks.