A comparison study of supervised learning techniques for the approximation of high dimensional functions and feedback control

TL;DR

This work tackles the challenge of high-dimensional value-function approximation for infinite-horizon optimal control by leveraging SDRE-derived data to train surrogate models. It systematically compares tensor-train based surrogates (including Functional TT, TT-Cross, and Block-Sparse TT), kernel methods, and neural networks across a range of high-dimensional problems, including regularity and control of the Allen–Cahn PDE. The results show that TT-based methods, particularly TT-Cross and related gradient-informed approaches, consistently achieve lower errors and favorable scaling with dimension, especially when the target function exhibits separability or regularity; kernel methods and neural networks remain flexible but often require more data or tuned stabilization to match TT performance. The findings underscore the value of exploiting problem structure in high-dimensional control tasks and clarify when each surrogate class is advantageous for generating accurate feedback laws in practice.

Abstract

Approximation of high dimensional functions is in the focus of machine learning and data-based scientific computing. In many applications, empirical risk minimisation techniques over nonlinear model classes are employed. Neural networks, kernel methods and tensor decomposition techniques are among the most popular model classes. We provide a numerical study comparing the performance of these methods on various high-dimensional functions with focus on optimal control problems, where the collection of the dataset is based on the application of the State-Dependent Riccati Equation.

Figures (5)

• Figure 1: Graphical representation of a TT representation of $c$ in four variables.
• Figure 2: Graphical representation of TT tensor train induced polynomial in four variables.
• Figure 3: Function and surrogate model for $d=16$ and $\lambda = (0,0.5,0)$ for the different methods on the plane $(x_1,x_2,0.5,\ldots,0.5)$. The localized kink is only fitted properly by the TT approximation.
• Figure 4: Visualization of the test error \ref{['eq:test_error']} (left) and the CPU time (right) for approximation of the value function \ref{['eq:vf_academic']} depending on the dimension $d$ ($x$-axis).
• Figure 5: Solution for the Allen-Cahn equation for the uncontrolled dynamics (top left), the TT Gradient Cross controlled dynamics (top right), the Kernel-Two Boxes controlled dynamics (bottom left) and NN controlled dynamics (bottom right)

Theorems & Definitions (3)

• Example 1: Block Sparsity
• Definition 1: Micha
• Remark 1