Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BAdaCost: Multi-class Boosting with Costs

Antonio Fernández-Baldera, José M. Buenaposada, Luis Baumela

TL;DR

BAdaCost introduces CMEL, a Cost-sensitive Multi-Class Exponential Loss, to extend boosting to cost-aware multi-class problems. By optimizing a stage-wise additive model with a cost matrix, BAdaCost unifies SAMME, PIBoost, and CS-AdaBoost as special cases and demonstrates notable gains on UCI datasets and in multi-view object detection (faces and cars). The approach yields explicit control over decision boundaries through the cost matrix and achieves practical efficiency by sharing a common multi-class representation, enabling faster training and cascade calibration. Overall, the framework provides a principled tool for asymmetric and cost-sensitive multiclass tasks with strong empirical performance and scalability advantages.

Abstract

We present BAdaCost, a multi-class cost-sensitive classification algorithm. It combines a set of cost-sensitive multi-class weak learners to obtain a strong classification rule within the Boosting framework. To derive the algorithm we introduce CMEL, a Cost-sensitive Multi-class Exponential Loss that generalizes the losses optimized in various classification algorithms such as AdaBoost, SAMME, Cost-sensitive AdaBoost and PIBoost. Hence unifying them under a common theoretical framework. In the experiments performed we prove that BAdaCost achieves significant gains in performance when compared to previous multi-class cost-sensitive approaches. The advantages of the proposed algorithm in asymmetric multi-class classification are also evaluated in practical multi-view face and car detection problems.

BAdaCost: Multi-class Boosting with Costs

TL;DR

BAdaCost introduces CMEL, a Cost-sensitive Multi-Class Exponential Loss, to extend boosting to cost-aware multi-class problems. By optimizing a stage-wise additive model with a cost matrix, BAdaCost unifies SAMME, PIBoost, and CS-AdaBoost as special cases and demonstrates notable gains on UCI datasets and in multi-view object detection (faces and cars). The approach yields explicit control over decision boundaries through the cost matrix and achieves practical efficiency by sharing a common multi-class representation, enabling faster training and cascade calibration. Overall, the framework provides a principled tool for asymmetric and cost-sensitive multiclass tasks with strong empirical performance and scalability advantages.

Abstract

We present BAdaCost, a multi-class cost-sensitive classification algorithm. It combines a set of cost-sensitive multi-class weak learners to obtain a strong classification rule within the Boosting framework. To derive the algorithm we introduce CMEL, a Cost-sensitive Multi-class Exponential Loss that generalizes the losses optimized in various classification algorithms such as AdaBoost, SAMME, Cost-sensitive AdaBoost and PIBoost. Hence unifying them under a common theoretical framework. In the experiments performed we prove that BAdaCost achieves significant gains in performance when compared to previous multi-class cost-sensitive approaches. The advantages of the proposed algorithm in asymmetric multi-class classification are also evaluated in practical multi-view face and car detection problems.
Paper Structure (27 sections, 4 theorems, 59 equations, 14 figures, 4 tables, 1 algorithm)

This paper contains 27 sections, 4 theorems, 59 equations, 14 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Boosting
  3. Binary Boosting
  4. Multi-class Boosting with multi-class responses
  5. Cost-sensitive Boosting
  6. Binary Cost-sensitive Boosting
  7. Multi-class Cost-sensitive Boosting
  8. BAdaCost
  9. Multi-class cost-sensitive margin
  10. BAdaCost: Boosting Adapted for Cost matrix
  11. Direct generalizations
  12. Experiments
  13. Minimizing costs: UCI repository
  14. Multi-class object detection
  15. BAdaCost positive class score computation
  16. ...and 12 more sections

Key Result

Lemma 1

Optimal $(\beta_m, {\hbox{\boldmath $\bf g$}}_m({\hbox{\boldmath $\bf x$}}))$ for CMEL Let ${\hbox{\boldmath $\bf C$}}$ be a cost matrix for a multi-class problem. Given the additive model ${\hbox{\boldmath $\bf f$}}_m ({\hbox{\boldmath $\bf x$}}) = {\hbox{\boldmath $\bf f$}}_{m-1}({\hbox{\boldmath is the same as the solution to where $S_j=\sum_{ \left\{ n : {\hbox{\boldmath $\bf g$}}(x_n) = l_n

Figures (14)

  • Figure 1: Comparison of ranks through the Bonferroni-Dunn test. BAdaCost's average rank is taken as reference. Algorithms significantly worse than our method for a significance level of $0.10$ are unified with a blue line.
  • Figure 2: Mean of the AFLW training images in each face view. From left to right: full right profile (view 1), half right profile (view 2), frontal face (view 3), half left profile (view 4) and full left profile (view 5).
  • Figure 3: Training with AFLW and validating with PASCAL (AFLW/PASCAL).
  • Figure 4: Training with AFLW and testing with AFW (AFLW/AFW experiment).
  • Figure 5: Training with AFLW and testing with FDDB.
  • ...and 9 more figures

Theorems & Definitions (4)

  • Lemma 1
  • Corollary 1: SAMME Zhu09 is a special case of BAdaCost
  • Corollary 2: CS-AdaBoost Masnadi11 is a special case of BAdaCost
  • Corollary 3: PIBoost Baldera14 is a special case of BAdaCost