Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Framework for Time-Varying Optimization via Derivative Estimation

Matteo Marchi, Jonathan Bunton, João Pedro Silvestre, Paulo Tabuada

TL;DR

This work treats time-varying optimization as a trajectory-tracking problem for the time-varying minimizer $x^*(\theta(t))$ and develops a robust framework that does not require direct knowledge of $\dot\theta$. It introduces a general dirtied-derivative estimator of order $k$, with explicit convergence and error bounds, and proves that interconnecting this estimator with a well-behaved continuous-time optimization algorithm preserves input-to-output stability (IOS). The result is an IOS guarantee for the time-varying problem when using the estimated derivative in place of the true time derivative, with performance improving as the estimator gain $\sigma$ increases (subject to noise and discretization limits). Empirical simulations demonstrate effective derivative tracking and improved loss trajectories for online Newton-like optimization when incorporating the estimator. Overall, the paper provides a principled, ISS-based methodology to adapt static continuous-time optimization algorithms to dynamic environments using derivative estimation.

Abstract

Optimization algorithms have a rich and fundamental relationship with ordinary differential equations given by its continuous-time limit. When the cost function varies with time -- typically in response to a dynamically changing environment -- online optimization becomes a continuous-time trajectory tracking problem. To accommodate these time variations, one typically requires some inherent knowledge about their nature such as a time derivative. In this paper, we propose a novel construction and analysis of a continuous-time derivative estimation scheme based on "dirty-derivatives", and show how it naturally interfaces with continuous-time optimization algorithms using the language of ISS (Input-to-State Stability). More generally, we show how a simple Lyapunov redesign technique leads to provable suboptimality guarantees when composing this estimator with any well-behaved optimization algorithm for time-varying costs.

A Framework for Time-Varying Optimization via Derivative Estimation

TL;DR

This work treats time-varying optimization as a trajectory-tracking problem for the time-varying minimizer and develops a robust framework that does not require direct knowledge of . It introduces a general dirtied-derivative estimator of order , with explicit convergence and error bounds, and proves that interconnecting this estimator with a well-behaved continuous-time optimization algorithm preserves input-to-output stability (IOS). The result is an IOS guarantee for the time-varying problem when using the estimated derivative in place of the true time derivative, with performance improving as the estimator gain increases (subject to noise and discretization limits). Empirical simulations demonstrate effective derivative tracking and improved loss trajectories for online Newton-like optimization when incorporating the estimator. Overall, the paper provides a principled, ISS-based methodology to adapt static continuous-time optimization algorithms to dynamic environments using derivative estimation.

Abstract

Optimization algorithms have a rich and fundamental relationship with ordinary differential equations given by its continuous-time limit. When the cost function varies with time -- typically in response to a dynamically changing environment -- online optimization becomes a continuous-time trajectory tracking problem. To accommodate these time variations, one typically requires some inherent knowledge about their nature such as a time derivative. In this paper, we propose a novel construction and analysis of a continuous-time derivative estimation scheme based on "dirty-derivatives", and show how it naturally interfaces with continuous-time optimization algorithms using the language of ISS (Input-to-State Stability). More generally, we show how a simple Lyapunov redesign technique leads to provable suboptimality guarantees when composing this estimator with any well-behaved optimization algorithm for time-varying costs.
Paper Structure (9 sections, 4 theorems, 37 equations, 5 figures)

This paper contains 9 sections, 4 theorems, 37 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Online Optimization
  4. Derivative Estimation
  5. Framework for Time-varying Optimization
  6. Experiments
  7. Derivative Tracking
  8. Time-Varying Optimization with Derivative Estimator
  9. Conclusions

Key Result

Lemma 1

Consider system eq:optimization_ode and suppose $g$ is such that for any fixed value of $\theta\in\Theta$ and its corresponding minimizer $x^*(\theta)$, system eq:optimization_ode is ISS with respect to $u$. Let $V(x,\theta)$ be a corresponding family of Lyapunov functions differentiable with respec renders eq:optimization_ode ISS with respect to the signal $d:\mathbb{T}\to\mathbb{R}^p$ when consi

Figures (5)

  • Figure 1: System \ref{['eq:dirty_der_recursive']} estimating the first three derivatives of a signal $U(s)$.
  • Figure 2: Plot of the first derivative of the signal $w(t)=\sin(5t-2)$ and its estimates for $\sigma\in\{5,20\}$.
  • Figure 3: Plot of the first derivative of the signal $w(t)=\sin(5t-2)$ and its estimates for $\sigma\in\{5,20\}$ when $w$ is corrupted by Gaussian noise.
  • Figure 4: Loss (over time) of the online Newton's method with ground truth (blue) and estimated (red, yellow) time derivative information.
  • Figure 5: Loss (over time) of the online Newton's method with noise-corrupted ground truth (blue) and estimated (red, yellow) time derivative information.

Theorems & Definitions (13)

  • Lemma 1
  • proof
  • Remark 1
  • Remark 2
  • Theorem 1
  • proof
  • Remark 3
  • Remark 4
  • Remark 5
  • Theorem 2
  • ...and 3 more