Deep Learning for Subspace Regression

TL;DR

The paper tackles the challenge of online approximating parameter-dependent linear subspaces by formulating subspace regression on the Grassmann manifold. It introduces Grassmann-invariant loss functions and a subspace-embedding strategy that trains networks to predict larger-than-needed subspaces, supported by theory showing smoother mappings and reduced complexity for certain problems. Through numerical experiments on parametric eigenproblems, Burgers-type ROMs, deflation, and two-grid methods, the authors demonstrate that embedding plus $L_2$-based losses yield substantial accuracy gains and robustness in high-dimensional settings, often outperforming classical interpolation. The work offers practical methodology for efficient ROM in high-dimensional parameter spaces and highlights promising directions for improving basis representations and operator learning in parametric PDEs.

Abstract

It is often possible to perform reduced order modelling by specifying linear subspace which accurately captures the dynamics of the system. This approach becomes especially appealing when linear subspace explicitly depends on parameters of the problem. A practical way to apply such a scheme is to compute subspaces for a selected set of parameters in the computationally demanding offline stage and in the online stage approximate subspace for unknown parameters by interpolation. For realistic problems the space of parameters is high dimensional, which renders classical interpolation strategies infeasible or unreliable. We propose to relax the interpolation problem to regression, introduce several loss functions suitable for subspace data, and use a neural network as an approximation to high-dimensional target function. To further simplify a learning problem we introduce redundancy: in place of predicting subspace of a given dimension we predict larger subspace. We show theoretically that this strategy decreases the complexity of the mapping for elliptic eigenproblems with constant coefficients and makes the mapping smoother for general smooth function on the Grassmann manifold. Empirical results also show that accuracy significantly improves when larger-than-needed subspaces are predicted. With the set of numerical illustrations we demonstrate that subspace regression can be useful for a range of tasks including parametric eigenproblems, deflation techniques, relaxation methods, optimal control and solution of parametric partial differential equations.

Figures (10)

• Figure 1: Selected results for eigenspace prediction: (a) Comparison of training time for losses $L_1(A, B)$, $L_2(A, B; z)$ from Theorem \ref{['th:loss_functions']}. On $D=2$ grid $N_x = N_y = 100$ we observe $L_2(A, B; z)$ scales better with dimension size; (b) Illustration of subspace embedding technique from Section \ref{['section:subspace_embedding']} for $D=2$ elliptic eigenproblem, prediction of first $10$ eigenvectors. Prediction of larger subspace manifestly improves accuracy and reduces generalisation gap; (c) Relative error for individual eigenvector predictions for the same problem as in (b) but trained with $\mathbb{Z}_2$-adjusted $l_2$ loss. Similarly to results of Theorem \ref{['th:parametric_eigenproblem']} we observe a steep increase of problem complexity with eigenvector number. See Section \ref{['subsection:eigenspaces']} for details.
• Figure 2: Relative errors for selected baselines. Label "subspace" refers to subspace regression. For the elliptic problem (a) subspace dimension of ROM methods is bounded by $100$, and for DeepPOD, and subspace regression -- by $40$. Oracle is omitted for the elliptic problem because it has perfect accuracy with $10$ basis functions. For Burgers equation subspace dimensions for all methods $\leq 50$. FFNO${}_{b}$ and DeepONet${}_{b}$ refer to an intrusive ROM with bases extracted from FFNO and DeepONet.
• Figure 3: Convergence results for iterative methods. Learned methods are marked with solid lines, and dashed lines correspond to iterative methods with optimal deflation and coarse-grid spaces, $M$ refers to subspace size.
• Figure 4: Example of subspace embedding detailed in Appendix \ref{['appendix:embedding_example']}.
• Figure 5: (a) Parametric family of ellipsoids passing through point $(4, 5)$. (b) Ellipsoid of minimal volume passing through $(4, 5)$. Note that the number of standard lattice points inside is approximately $4\times 5 = 20$, the error of approximation ($5$ in this case) is asymptotically small for ellipsoids of large volume. (c) Example of non-minimal ellipsoid passing through $(4, 5)$. In the non-minimal case, the number of standard lattice points inside an ellipsoid passing through a given point can be made arbitrarily large.

Theorems & Definitions (8)

• Theorem 1
• proof
• Theorem 2
• proof
• Theorem 3
• proof
• Lemma 1
• proof