Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning-Augmented Decentralized Online Convex Optimization in Networks

Pengfei Li, Jianyi Yang, Adam Wierman, Shaolei Ren

TL;DR

The average cost bound for LADO is proved, revealing the tradeoff between average performance and worst-case robustness and demonstrating the advantage of training the ML policy by explicitly considering the robustness requirement.

Abstract

This paper studies decentralized online convex optimization in a networked multi-agent system and proposes a novel algorithm, Learning-Augmented Decentralized Online optimization (LADO), for individual agents to select actions only based on local online information. LADO leverages a baseline policy to safeguard online actions for worst-case robustness guarantees, while staying close to the machine learning (ML) policy for average performance improvement. In stark contrast with the existing learning-augmented online algorithms that focus on centralized settings, LADO achieves strong robustness guarantees in a decentralized setting. We also prove the average cost bound for LADO, revealing the tradeoff between average performance and worst-case robustness and demonstrating the advantage of training the ML policy by explicitly considering the robustness requirement.

Learning-Augmented Decentralized Online Convex Optimization in Networks

TL;DR

The average cost bound for LADO is proved, revealing the tradeoff between average performance and worst-case robustness and demonstrating the advantage of training the ML policy by explicitly considering the robustness requirement.

Abstract

This paper studies decentralized online convex optimization in a networked multi-agent system and proposes a novel algorithm, Learning-Augmented Decentralized Online optimization (LADO), for individual agents to select actions only based on local online information. LADO leverages a baseline policy to safeguard online actions for worst-case robustness guarantees, while staying close to the machine learning (ML) policy for average performance improvement. In stark contrast with the existing learning-augmented online algorithms that focus on centralized settings, LADO achieves strong robustness guarantees in a decentralized setting. We also prove the average cost bound for LADO, revealing the tradeoff between average performance and worst-case robustness and demonstrating the advantage of training the ML policy by explicitly considering the robustness requirement.
Paper Structure (42 sections, 11 theorems, 77 equations, 7 figures, 7 tables, 1 algorithm)

This paper contains 42 sections, 11 theorems, 77 equations, 7 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Performance Metrics
  4. Application Examples
  5. LADO-Lin: Linearly Combining ML Advice and Expert Advice
  6. Local Information Availability
  7. Algorithm Design
  8. Performance Analysis
  9. LADO: Adaptively Combining ML Advice and Expert Advice
  10. Algorithm Design
  11. Designing a Robust Action Set
  12. Spatial Cost Decomposition
  13. Robust Action Sets via Reservation Costs
  14. Performance Bounds for LADO
  15. $\lambda$-Competitiveness
  16. ...and 27 more sections

Key Result

Theorem 3.1

For any $\gamma \in [0,1]$, the average cost of LADO-Lin is upper bounded by where $\text{AVG}(\tilde{\pi})$ and $\text{AVG}(\pi^\dagger)$ are the average costs of the ML policy and expert over the distribution $g_{1:T}\sim\mathcal{P}_{g_{1:T}}$, respectively.

Figures (7)

  • Figure 1: The evaluation of LADO and baseline algorithms in terms of the node, temporal and spatial costs, with various graph topologies. By default, the competitiveness requirement $\lambda$ is set to 1 in LADO for all the graphs.
  • Figure 2: Impact of graph density and competitiveness requirement $\lambda$ on the overall cost of LADO, along with the additional cost (regret) associated with the projection process compared to the ML policy.
  • Figure 3: The total cost distribution of LADO and baseline algorithms with various graph topologies (15-node network). By default, the competitiveness requirement $\lambda$ is set to 1 in LADO for all graphs.
  • Figure 4: The comparison of total cost and competitive ratio distribution between LADO and other baseline algorithms for a 15-node network.
  • Figure 5: Impact of graph sizes and competitiveness requirement $\lambda$ on the overall cost of LADO, along with the additional cost (regret) incurred by the projection process compared to the ML policy. The overall cost and regret are normalized by the number of nodes for a consistent comparison across different graph sizes.
  • ...and 2 more figures

Theorems & Definitions (18)

  • Definition 2.4: Average cost
  • Definition 2.5: $\lambda$-competitive to $\pi^{\dagger}$
  • Theorem 3.1: Average cost of LADO-Lin
  • Theorem 3.2
  • Definition 3.3: Robustness-consistency
  • Corollary 3.4: Robustness-consistency of LADO-Lin
  • Proposition 4.1: Anytime $\lambda$-competitiveness
  • proof
  • Theorem 5.1
  • Theorem 5.2
  • ...and 8 more