Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Specifying and Solving Robust Empirical Risk Minimization Problems Using CVXPY

Eric Luxenberg, Dhruv Malik, Yuanzhi Li, Aarti Singh, Stephen Boyd

TL;DR

CVXPY can be used to automate this dualization procedure in a user-friendly manner and allows practitioners to specify and solve robust ERM problems with a general class of convex losses, capturing many standard regression and classification problems.

Abstract

We consider robust empirical risk minimization (ERM), where model parameters are chosen to minimize the worst-case empirical loss when each data point varies over a given convex uncertainty set. In some simple cases, such problems can be expressed in an analytical form. In general the problem can be made tractable via dualization, which turns a min-max problem into a min-min problem. Dualization requires expertise and is tedious and error-prone. We demonstrate how CVXPY can be used to automate this dualization procedure in a user-friendly manner. Our framework allows practitioners to specify and solve robust ERM problems with a general class of convex losses, capturing many standard regression and classification problems. Users can easily specify any complex uncertainty set that is representable via disciplined convex programming (DCP) constraints.

Specifying and Solving Robust Empirical Risk Minimization Problems Using CVXPY

TL;DR

CVXPY can be used to automate this dualization procedure in a user-friendly manner and allows practitioners to specify and solve robust ERM problems with a general class of convex losses, capturing many standard regression and classification problems.

Abstract

We consider robust empirical risk minimization (ERM), where model parameters are chosen to minimize the worst-case empirical loss when each data point varies over a given convex uncertainty set. In some simple cases, such problems can be expressed in an analytical form. In general the problem can be made tractable via dualization, which turns a min-max problem into a min-min problem. Dualization requires expertise and is tedious and error-prone. We demonstrate how CVXPY can be used to automate this dualization procedure in a user-friendly manner. Our framework allows practitioners to specify and solve robust ERM problems with a general class of convex losses, capturing many standard regression and classification problems. Users can easily specify any complex uncertainty set that is representable via disciplined convex programming (DCP) constraints.
Paper Structure (17 sections, 10 equations, 2 figures)

This paper contains 17 sections, 10 equations, 2 figures.

Table of Contents

  1. Robust empirical risk minimization
  2. Examples.
  3. Solving RERM problems
  4. Analytical cases.
  5. Dualization.
  6. Automatic dualization via CVXPY.
  7. Previous and related work
  8. Robust optimization and saddle problems.
  9. RERM.
  10. Reformulating the RERM problem
  11. Loss $f$ is non-increasing.
  12. Loss $f$ is non-decreasing on ${\hbox{\bf R}}_+$ and $f(a) = f(\vert a \vert)$.
  13. CVXPY code.
  14. Example
  15. Data.
  16. ...and 2 more sections

Figures (2)

  • Figure 1: A visualization of $D_i$ and $S_i$ for a particular rental $i$. The large disk around the red dot corresponds to $D_i$. The square containing the blue dot corresponds to $S_i$. The overlap of the square and the disk corresponds to $D_i \cap S_i$.
  • Figure 2: Excess test error in the Airbnb price prediction experiment.