High-order Regularization for Machine Learning and Learning-based Control

TL;DR

High-order regularization (HR) introduces a novel inverse-mapping view of regularization for neural networks, providing convergence guarantees and calculable error bounds. By expressing the inverse of the information matrix through a truncated matrix-power series, with HR solution hat_beta_{hr} = (H^T H + R)^{-1} sum_{i=0}^c F(R)^i H^T Y under rho(F(R)) < 1, HR unifies L2 as the c=0 special case and enables contraction toward an optimal solution. The framework yields theoretically grounded guidance for selecting the regularization matrix R and order c to balance bias and generalizability, and it accommodates incremental and scalable neural networks. Empirically, applying HR to regularized extreme learning machines in a cart-pole control task achieves superior stability and generalizability compared with Q-networks and EQLM, illustrating its practical impact for learning-based control and explainable learning.

Abstract

The paper proposes a novel regularization procedure for machine learning. The proposed high-order regularization (HR) provides new insight into regularization, which is widely used to train a neural network that can be utilized to approximate the action-value function in general reinforcement learning problems. The proposed HR method ensures the provable convergence of the approximation algorithm, which makes the much-needed connection between regularization and explainable learning using neural networks. The proposed HR method theoretically demonstrates that regularization can be regarded as an approximation in terms of inverse mapping with explicitly calculable approximation error, and the $L_2$ regularization is a lower-order case of the proposed method. We provide lower and upper bounds for the error of the proposed HR solution, which helps build a reliable model. We also find that regularization with the proposed HR can be regarded as a contraction. We prove that the generalizability of neural networks can be maximized with a proper regularization matrix, and the proposed HR is applicable for neural networks with any mapping matrix. With the theoretical explanation of the extreme learning machine for neural network training and the proposed high-order regularization, one can better interpret the output of the neural network, thus leading to explainable learning. We present a case study based on regularized extreme learning neural networks to demonstrate the application of the proposed HR and give the corresponding incremental HR solution. We verify the performance of the proposed HR method by solving a classic control problem in reinforcement learning. The result demonstrates the superior performance of the method with significant enhancement in the generalizability of the neural network.

Figures (13)

• Figure 1: Regression problems (left) and classification problems (right) in machine learning. The dots represent the training data samples. The black lines denote regularized models with less complexity while the blue lines are some over-fitting models.
• Figure 2: The ELM architecture.
• Figure 3: The agent-environment interface in RL.
• Figure 4: The relationship between $\text{Obj}(R)$ and $\bar{\mu}$ for some selections. $R=\bar{\mu}$ in the left figure, and $\lambda_{R,i}= \sigma_{1} -\sigma_{i} +\bar{\mu}$ in the right, where $\sigma_{i}(\sigma_{i}\geq\sigma_{i+1},i=1,\cdots, n)$ is the singular value of $H^TH$. There are different objective functions $\text{Obj}(R)$ with respect to different regularization order $c\in \{0,\cdots, 5\}$ for the two selections of $R$.
• Figure 5: Illustration of regularization. The largest circle indicates the range of possible solutions provided by the original neural network with the information matrix $H^TH$ for a given training data set. The center of the largest circle is the theoretical optimal solution. The smaller circles show the regularization solutions under different regularization matrices $R$. The radius of the dotted circle is the bias for a given regularization order $c$. A regularization solution with a proper $R$ provides the trade-off between the generalizability and the estimation bias. Utilizing the proposed HR method, as the regularization matrix moves toward the optimal parameter, the regularization solution approaches the optimal solution while pursuing the smallest condition number of the new matrix $H^TH+R$, in other words, maximizing the generalizability of the neural network.

Theorems & Definitions (15)

• Theorem 1
• proof
• proof
• Lemma 1
• proof
• Theorem 2
• proof
• Proposition 1
• proof
• Proposition 2