Online Optimization with Unknown Time-varying Parameters

TL;DR

This work addresses online optimization where cost parameters $z(t)$ evolve according to unknown linear dynamics $z(t+1)=A z(t)$. It introduces a three-stage, gradient-measurement-driven identification pipeline that first estimates $A$ up to a similarity, then recovers the exact coordinate transformation to obtain $A$ and $z(t)$, and finally uses the identified dynamics to track the time-varying minimizer of $f(x,z(t))$. The approach relies on subspace identification from Hankel-formed gradient data and explicit rank/observability conditions to guarantee finite-sample identifiability and exact recovery of the dynamics. Numerical experiments across quadratic, polynomial, and non-polynomial cost functions demonstrate accurate tracking and finite-time convergence, outperforming static gradient descent that ignores the time variation. Overall, the paper provides a principled framework for online time-varying optimization with unknown evolving parameters and lays groundwork for extensions to broader nonlinear settings, noise robustness, and constraints.

Abstract

In this paper, we study optimization problems where the cost function contains time-varying parameters that are unmeasurable and evolve according to linear, yet unknown, dynamics. We propose a solution that leverages control theoretic tools to identify the dynamics of the parameters, predict their evolution, and ultimately compute a solution to the optimization problem. The identification of the dynamics of the time-varying parameters is done online using measurements of the gradient of the cost function. This system identification problem is not standard, since the output matrix is known and the dynamics of the parameters must be estimated in the original coordinates without similarity transformations. Interestingly, our analysis shows that, under mild conditions that we characterize, the identification of the parameters dynamics and, consequently, the computation of a time-varying solution to the optimization problem, requires only a finite number of measurements of the gradient of the cost function. We illustrate the effectiveness of our algorithm on a series of numerical examples.

Figures (4)

• Figure 1: This figure shows the predicted (solid blue line) and the optimal solution (dashed red line) considering a quadratic cost function for the setting described in Section \ref{['subsec:\n time-varying quadratic term']} as a function of time. Panels (a) and (b) shows the evolution of the first and the second component of the predicted and the optimal solution, respectively. The data $X_0$ and $Y_0$ in \ref{['eq: data']} are collected when $t\!\in\![0,8]$, where $[x_1(t),x_2(t)]^{\mathsf{T}}= [\sqrt{2}/2, \sqrt{2}/2]^{\mathsf{T}}$ and the data $X_1$ and $Y_1$ in \ref{['eq: data']} are collected when $t\! \in\! [9, 26]$, where $x(t)$ is obtained using static gradient descent \ref{['eq: for comparison']}. For $t>26$, we predict the optimal solution using \ref{['eq:quad_sol']}. We observe that the predicted value converges to the optimal solution for $t>26$.
• Figure 2: The figure shows a comparison of our proposed solution (solid blue line) with the static gradient descent algorithm \ref{['eq:\n for comparison']} (dashed red line). Panel (a) and (b) shows the error in estimating the optimal solution for a quadratic cost function and a higher order polynomial, respectively. We observe that the error decays to zero for $t>N$ using our solution proposed in Section \ref{['subsec:\n time-varying quadratic term']} and \ref{['subsec:\n time-varying polynomial term']}. However, for the static gradient descent \ref{['eq: for comparison']}, we observe that the error diverges.
• Figure 3: This figure shows the predicted (solid blue line) and the optimal solution (dashed red line) considering a third-order polynomial cost function for the setting described in Section \ref{['subsec:\n time-varying polynomial term']} as a function of time. Panels (a) and (b) shows the evolution of the first and the second component of the predicted and the optimal solution, respectively.The data $X_0$ and $Y_0$ in \ref{['eq: data']} are collected when $t\!\in\![0,18]$, where $[x_1(t),x_2(t)]^{\mathsf{T}}=[\sqrt{2}/2, \sqrt{2}/2]^{\mathsf{T}}$ and the data $X_1$ and $Y_1$ in \ref{['eq: data']} are collected when $t\! \in\! [19, 60]$, where $x(t)$ is obtained using \ref{['eq: for comparison']}. For $t>26$, we predict the optimal solution using algorithm \ref{['Algo1']}. We observe that the predicted value converges to the optimal solution for $t>60$.
• Figure 4: The figure in panel (a) shows the predicted (solid blue line) and the optimal solution (dashed red line) considering a non-polynomial cost function for the setting described in Section \ref{['section:non-polynomial']} as a function of time. The data $X_0$ and $Y_0$ in \ref{['eq: data']} are collected when $t\!\in\![0,6]$, where $[x_1(t),x_2(t)]^{\mathsf{T}}=[\sqrt{2}/2, \sqrt{2}/2]^{\mathsf{T}}$ and the data $X_1$ and $Y_1$ in \ref{['eq: data']} are collected when $t\! \in\! [7, 30]$, where $x(t)$ is obtained using \ref{['eq: for comparison']}. For $t>26$, we predict the optimal solution using algorithm \ref{['Algo1']}. The figure in panel (b) shows the error in computing the optimal solution, using our solution (solid blue line) and static gradient descent \ref{['eq: for comparison']} (dashed red line). We observe that unlike the static gradient descent algorithm \ref{['eq: for comparison']}, the predicted value converges to the optimal solution using our proposed solution.

Theorems & Definitions (5)

• Example 1
• Theorem 3.1
• Theorem 3.2
• Lemma E.1.1
• Lemma E.1.2