Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scalable Kernel Inverse Optimization

Youyuan Long, Tolga Ok, Pedro Zattoni Scroccaro, Peyman Mohajerin Esfahani

TL;DR

This paper extends the hypothesis class of IO objective functions to a reproducing kernel Hilbert space (RKHS), thereby enhancing feature representation to an infinite-dimensional space and demonstrates that a variant of the representer theorem holds for a specific training loss, allowing the reformulation of the problem as a finite-dimensional convex optimization program.

Abstract

Inverse Optimization (IO) is a framework for learning the unknown objective function of an expert decision-maker from a past dataset. In this paper, we extend the hypothesis class of IO objective functions to a reproducing kernel Hilbert space (RKHS), thereby enhancing feature representation to an infinite-dimensional space. We demonstrate that a variant of the representer theorem holds for a specific training loss, allowing the reformulation of the problem as a finite-dimensional convex optimization program. To address scalability issues commonly associated with kernel methods, we propose the Sequential Selection Optimization (SSO) algorithm to efficiently train the proposed Kernel Inverse Optimization (KIO) model. Finally, we validate the generalization capabilities of the proposed KIO model and the effectiveness of the SSO algorithm through learning-from-demonstration tasks on the MuJoCo benchmark.

Scalable Kernel Inverse Optimization

TL;DR

This paper extends the hypothesis class of IO objective functions to a reproducing kernel Hilbert space (RKHS), thereby enhancing feature representation to an infinite-dimensional space and demonstrates that a variant of the representer theorem holds for a specific training loss, allowing the reformulation of the problem as a finite-dimensional convex optimization program.

Abstract

Inverse Optimization (IO) is a framework for learning the unknown objective function of an expert decision-maker from a past dataset. In this paper, we extend the hypothesis class of IO objective functions to a reproducing kernel Hilbert space (RKHS), thereby enhancing feature representation to an infinite-dimensional space. We demonstrate that a variant of the representer theorem holds for a specific training loss, allowing the reformulation of the problem as a finite-dimensional convex optimization program. To address scalability issues commonly associated with kernel methods, we propose the Sequential Selection Optimization (SSO) algorithm to efficiently train the proposed Kernel Inverse Optimization (KIO) model. Finally, we validate the generalization capabilities of the proposed KIO model and the effectiveness of the SSO algorithm through learning-from-demonstration tasks on the MuJoCo benchmark.

Paper Structure

This paper contains 20 sections, 4 theorems, 26 equations, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notation
  3. Preliminaries
  4. Inverse Optimization
  5. The Kernel Method
  6. Kernel Inverse Optimization
  7. Sequential Selection Optimization
  8. Heuristic for choosing coordinates
  9. Warm-up trick for improved initialization
  10. Numerical Experiments
  11. Performance Evaluation
  12. Evaluation for KIO.
  13. Evaluation for SSO.
  14. Ablation Studies
  15. Conclusion and Limitations
  16. ...and 5 more sections

Key Result

Proposition 1

For the hypothesis function and feasible set in eq:hypothesis, the optimization program eq:loss_minimization is equivalent to

Figures (3)

  • Figure 1: Final Objective Function Value and Score (average return over 100 evaluations) for SCS ocpb:16 and SSO (20 iterations for all tasks) algorithms. The ultimate Objective Function Values of the two algorithms are nearly identical, yet across the majority of tasks, SSO achieves a slightly higher score compared to SCS.
  • Figure 2: Convergence curves for SSO.
  • Figure 3: Convergence curves on the MuJoCo Hopper task with the first 5k data points from the D4RL Hopper-expert dataset. The vertical axis represents the difference between the current objective function value and the optimal value. Sequential Selection Optimization (orange) exhibits the fastest convergence rate.

Theorems & Definitions (7)

  • Proposition 1: LMI reformulation akhtar2021learning
  • Theorem 1: Kernel reformulation
  • proof
  • Remark 1: A potential variant of representer theorem
  • Corollary 1: Kernel reformulation for $\theta_{uu} = I_n$
  • Proposition 2
  • proof