ELM-DeepONets: Backpropagation-Free Training of Deep Operator Networks via Extreme Learning Machines

TL;DR

ELM-DeepONet introduces a backpropagation-free training scheme for DeepONets by embedding Extreme Learning Machine ideas into the trunk-branch operator framework. By fixing the trunk and using a learnable matrix $\mathbf{W}$ to fuse branch outputs into trunk coefficients, the method reduces training to a fast least-squares problem with the solution $\widehat{\mathbf{W}} = \mathbf{T}^\dagger \mathbf{G} \mathbf{B}^\dagger$. Extensive experiments on nonlinear ODEs, Darcy flow, and a reaction-diffusion inverse source problem show that ELM-DeepONet achieves competitive or superior accuracy while drastically reducing training time compared to vanilla DeepONet. This work offers a scalable, efficient alternative for operator learning in scientific computing, with practical guidance on hyperparameters and potential extensions to physics-informed variants.

Abstract

Deep Operator Networks (DeepONets) are among the most prominent frameworks for operator learning, grounded in the universal approximation theorem for operators. However, training DeepONets typically requires significant computational resources. To address this limitation, we propose ELM-DeepONets, an Extreme Learning Machine (ELM) framework for DeepONets that leverages the backpropagation-free nature of ELM. By reformulating DeepONet training as a least-squares problem for newly introduced parameters, the ELM-DeepONet approach significantly reduces training complexity. Validation on benchmark problems, including nonlinear ODEs and PDEs, demonstrates that the proposed method not only achieves superior accuracy but also drastically reduces computational costs. This work offers a scalable and efficient alternative for operator learning in scientific computing.

Figures (7)

• Figure 1: An illustration of ELM with SLFN. First hidden layer parameter $W_1$ is initialized randomly and fixed, while the output layer parameter $W_2$ is determined by solving a least squares problem.
• Figure 2: An illustration of DeepONet, with the picture adapted from lu2021learning.
• Figure 3: An illustration of the proposed ELM-DeepONets. The parameters of the branch and trunk networks are fixed during training, while an additional learnable parameter $W \in \mathbb{R}^{p_2\times p_1}$ is introduced to generate the output $G(u)(y)$.
• Figure 4: Numerical results for the antiderivative example on a single test sample demonstrate that DeepONet fails to capture the exact solution, whereas both ELM-DeepONets, one with fixed neural networks and the other with sinusoidal basis functions, accurately predict the true target function.
• Figure 5: An illustration of sinusoidal basis functions for the Darcy Flow example. Fixed basis functions are used as an alternative to the trunk network. The solid lines are included solely for visual consistency with the trunk network and do not represent weights in this context.