Data-Efficient Extremum-Seeking Control Using Kernel-Based Function Approximation

TL;DR

The paper addresses the costly measurement burden of extremum-seeking control by introducing kernel-based, data-efficient ESC (KB-ESC) that online builds a local approximation $m_k(\theta)$ of the unknown steady-state map $f(\theta)$ using data collected during regular ESC operation. A descent direction is pursued via $\theta^+ = \theta - \mu \nabla m_k(\theta)$ only when a guaranteed decrease can be established through Armijo-type bounds computed from a kernel-based SOCP, otherwise a standard update is used and data are augmented for the next iteration. Theoretical stability results show convergence of the optimizer state to a neighborhood of the minimizer set $\mathcal{C}$, and simulations on a nonlinear, multi-input system demonstrate a reduction in both measurements (about 40%) and updates (about 28%) without compromising performance. The approach leverages the Representer theorem and RKHS properties to obtain tractable bounds and integrates data reuse with ESC for practical, resource-efficient optimization in unknown dynamical systems.

Abstract

Existing extremum-seeking control (ESC) approaches typically rely on applying repeated perturbations to input parameters and performing measurements of the corresponding performance output. The required separation between the different timescales in the ESC loop means that performing these measurements can be time-consuming. Moreover, performing these measurements can be costly in practice, e.g., due to the use of resources. With these challenges in mind, it is desirable to reduce the number of measurements needed to optimize performance. Therefore, in this work, we present a sampled-data ESC approach aimed at reducing the number of measurements that need to be performed. In the proposed approach, we use input-output data obtained during regular operation of the extremum-seeking controller to construct online an approximation of the system's underlying cost function. By using this approximation to perform parameter updates when a decrease in the cost can be guaranteed, instead of performing additional measurements to perform this update, we make more efficient use of data collected during regular operation of the extremum-seeking controller. As a result, we indeed obtain a reduction in the required number of measurements to achieve optimization. We provide a stability analysis of the novel sampled-data ESC approach, and demonstrate the benefits of the synergy between kernel-based function approximation and standard ESC in simulation on a multi-input dynamical system.

Figures (5)

• Figure 1: Block scheme illustrating kernel-based extremum-seeking control.
• Figure 2: The optimizer state $\widehat{\theta}$ for both ESC approaches as a function of the number of measurements. The proposed kernel-based approach requires only 60 measurements to reach the same neighborhood of the optimum that the standard approach reaches with 100 measurements.
• Figure 3: The optimizer state $\widehat{\theta}$ for both ESC approaches as a function of the number of update steps. The third, fifth, and seventh update in the proposed kernel-based approach are kernel-based update steps, which do not require additional measurements to be performed to perform the update and allow searching for a suitable optimizer gain, resulting in needing only 18 update steps to reach the same neighborhood of the optimum as the standard approach does with 25 update steps.
• Figure 4: Contour plot of the steady-state input-output map $f(\theta)$, along with inputs $\widetilde{\theta}_i$ applied during measurements in both approaches. These applied inputs lie close to the line passing through the initial optimizer state $\widehat{\theta}_0$ and the minimizer $\theta^*$ (shown in green).
• Figure 5: Cross-section of the input-output map $f(\theta)$ as well as the approximation $m_k(\theta)$ at different kernel-based update steps, along the search direction (cf. green line in Figure \ref{['fig:contour']}). $m_k(\theta)$ quickly improves, allowing large kernel-based update steps to be taken from the circle to the square.