Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How to Learn a Star: Binary Classification with Starshaped Polyhedral Sets

Marie-Charlotte Brandenburg, Katharina Jochemko

TL;DR

This work develops a geometric framework for binary classification where decision boundaries are star-shaped polyhedral sets on a fixed simplicial fan. It reveals that classifier behavior partitions parameter space into data-arrangement chambers, establishing a VC dimension equal to the fan's number of generators and detailing the star-convex structure of sublevel sets for 0/1 loss, as well as the concave, convex-superlevel structure for the log-likelihood loss. The paper extends the model to translations and joint shape-translation, showing semialgebraic (and hence richly structured) extended loss landscapes along with VC bounds for the generalized parameter space. Preliminary 2D experiments illustrate performance relative to standard methods and highlight the practical importance of choosing the underlying fan and translation parameters.

Abstract

We consider binary classification restricted to a class of continuous piecewise linear functions whose decision boundaries are (possibly nonconvex) starshaped polyhedral sets, supported on a fixed polyhedral simplicial fan. We investigate the expressivity of these function classes and describe the combinatorial and geometric structure of the loss landscape, most prominently the sublevel sets, for two loss-functions: the 0/1-loss (discrete loss) and a log-likelihood loss function. In particular, we give explicit bounds on the VC dimension of this model, and concretely describe the sublevel sets of the discrete loss as chambers in a hyperplane arrangement. For the log-likelihood loss, we give sufficient conditions for the optimum to be unique, and describe the geometry of the optimum when varying the rate parameter of the underlying exponential probability distribution.

How to Learn a Star: Binary Classification with Starshaped Polyhedral Sets

TL;DR

This work develops a geometric framework for binary classification where decision boundaries are star-shaped polyhedral sets on a fixed simplicial fan. It reveals that classifier behavior partitions parameter space into data-arrangement chambers, establishing a VC dimension equal to the fan's number of generators and detailing the star-convex structure of sublevel sets for 0/1 loss, as well as the concave, convex-superlevel structure for the log-likelihood loss. The paper extends the model to translations and joint shape-translation, showing semialgebraic (and hence richly structured) extended loss landscapes along with VC bounds for the generalized parameter space. Preliminary 2D experiments illustrate performance relative to standard methods and highlight the practical importance of choosing the underlying fan and translation parameters.

Abstract

We consider binary classification restricted to a class of continuous piecewise linear functions whose decision boundaries are (possibly nonconvex) starshaped polyhedral sets, supported on a fixed polyhedral simplicial fan. We investigate the expressivity of these function classes and describe the combinatorial and geometric structure of the loss landscape, most prominently the sublevel sets, for two loss-functions: the 0/1-loss (discrete loss) and a log-likelihood loss function. In particular, we give explicit bounds on the VC dimension of this model, and concretely describe the sublevel sets of the discrete loss as chambers in a hyperplane arrangement. For the log-likelihood loss, we give sufficient conditions for the optimum to be unique, and describe the geometry of the optimum when varying the rate parameter of the underlying exponential probability distribution.
Paper Structure (39 sections, 21 theorems, 45 equations, 12 figures, 1 algorithm)

This paper contains 39 sections, 21 theorems, 45 equations, 12 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Our contributions
  3. Geometry of the parameter space.
  4. Geometry of sublevel sets.
  5. Extended parameter spaces.
  6. Limitations
  7. Description of the model
  8. Polyhedral geometry and stars
  9. Loss functions
  10. Geometry of the parameter space
  11. Geometry of the 0/1-loss
  12. Log-likelihood loss
  13. Geometry of the generalized parameter space
  14. Translations of a fixed star
  15. Translations and transformations of the star together
  16. ...and 24 more sections

Key Result

Proposition 1

For every vector $\mathbf{a}=(a_1,\ldots, a_n)\in \mathbb{R}^n$ there is a unique function $f^\Delta _\mathbf{a} : \mathbb{R}^d \rightarrow \mathbb{R}$ such that $f^\Delta_\mathbf{a}(\mathbf{v}_i) = a_i$ for $1\le i\le n$ and the restriction $f^\Delta _\mathbf{a} |_C$ is linear for any cone $C\in \D

Figures (12)

  • Figure 1: Two identical classifications of 3 points by different starshaped polyhedral sets, supported on the same polyhedral fan with $8$ generators in $\mathbb{R}^2$.
  • Figure 2: An example of a $1$-dimensional dataset, perfectly classified by a star supported on a fan with $n=2$ rays, and the level sets of the two loss functions in parameter space $\mathbb{R}^2_{>0}$. This example is explained in detail in \ref{['sec:appendix-examples-arrangement']}.
  • Figure 3: Synthetic data and accuracy of classification for tested models.
  • Figure 4: The functions $[\mathbf{x}]^\diamond$ and $[\mathbf{x}]^B$ restricted to the full-dimensional cones of the $2$-dimensional fans from \ref{['ex:kite-fans', 'ex:star-fans']}. $\mathbf{0}_k$ denotes the $k$-dimensional $0$-vector.
  • Figure 5: The data arrangement and dataset from \ref{['ex:data-arrangement']}.
  • ...and 7 more figures

Theorems & Definitions (46)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Proposition 1
  • Theorem 3.1
  • Proposition 2
  • Corollary 1
  • Theorem 3.2
  • Theorem 3.3
  • ...and 36 more