Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust semi-parametric mixtures of linear experts using the contaminated Gaussian distribution

Peterson Mambondimumwe, Sphiwe B. Skhosana, Najmeh Nakhaei Rad

TL;DR

This work tackles robust modeling of finite mixtures of linear regressions in the presence of mild outliers by introducing contaminated Gaussian components within a mixture-of-experts framework. The authors develop both a fully parametric CG-MoLE and a semi-parametric S-CG-MoLE, where the gating proportions are non-parametric functions estimated via local-likelihood kernel methods (restricted to a single covariate to mitigate the curse of dimensionality). Estimation relies on an ECM algorithm with an E-step and two CM-steps, and the non-parametric gating is obtained through a localized kernel regression that blends with the likelihood-based updates. The approach yields simultaneous clustering and outlier detection, demonstrated via extensive simulations across various contamination scenarios and an applied tone-perception dataset, where the robust models outperform Gaussian rivals and provide interpretable gating and outlier labels. This framework advances robust model-based clustering with flexible gating and has practical impact for regression mixtures in applications prone to mild outliers and non-Gaussian error behavior.

Abstract

Semi- and non-parametric mixture of regressions are a very useful flexible class of mixture of regressions in which some or all of the parameters are non-parametric functions of the covariates. These models are, however, based on the Gaussian assumption of the component error distributions. Thus, their estimation is sensitive to outliers and heavy-tailed error distributions. In this paper, we propose semi- and non-parametric contaminated Gaussian mixture of regressions to robustly estimate the parametric and/or non-parametric terms of the models in the presence of mild outliers. The virtue of using a contaminated Gaussian error distribution is that we can simultaneously perform model-based clustering of observations and model-based outlier detection. We propose two algorithms, an expectation-maximization (EM)-type algorithm and an expectation-conditional-maximization (ECM)-type algorithm, to perform maximum likelihood and local-likelihood kernel estimation of the parametric and non-parametric of the proposed models, respectively. The robustness of the proposed models is examined using an extensive simulation study. The practical utility of the proposed models is demonstrated using real data.

Robust semi-parametric mixtures of linear experts using the contaminated Gaussian distribution

TL;DR

This work tackles robust modeling of finite mixtures of linear regressions in the presence of mild outliers by introducing contaminated Gaussian components within a mixture-of-experts framework. The authors develop both a fully parametric CG-MoLE and a semi-parametric S-CG-MoLE, where the gating proportions are non-parametric functions estimated via local-likelihood kernel methods (restricted to a single covariate to mitigate the curse of dimensionality). Estimation relies on an ECM algorithm with an E-step and two CM-steps, and the non-parametric gating is obtained through a localized kernel regression that blends with the likelihood-based updates. The approach yields simultaneous clustering and outlier detection, demonstrated via extensive simulations across various contamination scenarios and an applied tone-perception dataset, where the robust models outperform Gaussian rivals and provide interpretable gating and outlier labels. This framework advances robust model-based clustering with flexible gating and has practical impact for regression mixtures in applications prone to mild outliers and non-Gaussian error behavior.

Abstract

Semi- and non-parametric mixture of regressions are a very useful flexible class of mixture of regressions in which some or all of the parameters are non-parametric functions of the covariates. These models are, however, based on the Gaussian assumption of the component error distributions. Thus, their estimation is sensitive to outliers and heavy-tailed error distributions. In this paper, we propose semi- and non-parametric contaminated Gaussian mixture of regressions to robustly estimate the parametric and/or non-parametric terms of the models in the presence of mild outliers. The virtue of using a contaminated Gaussian error distribution is that we can simultaneously perform model-based clustering of observations and model-based outlier detection. We propose two algorithms, an expectation-maximization (EM)-type algorithm and an expectation-conditional-maximization (ECM)-type algorithm, to perform maximum likelihood and local-likelihood kernel estimation of the parametric and non-parametric of the proposed models, respectively. The robustness of the proposed models is examined using an extensive simulation study. The practical utility of the proposed models is demonstrated using real data.
Paper Structure (18 sections, 37 equations, 6 figures, 6 tables, 2 algorithms)

This paper contains 18 sections, 37 equations, 6 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Methodology
  3. Contaminated Gaussian mixture of linear experts
  4. E-step
  5. CM-step 1
  6. CM-step 2
  7. Semi-parametric mixtures of contaminated Gaussian linear experts
  8. Model definition
  9. Model estimation
  10. E-step
  11. CM-step 1
  12. CM-step 2
  13. Model selection
  14. Model-based clustering
  15. Simulation Study
  16. ...and 3 more sections

Figures (6)

  • Figure 1: Scatter plot of the tone perception data
  • Figure 2: Estimated mixing proportions for the first component of all six fitted models on the tone perception data
  • Figure 3: Estimated models on the tone perception data: (a) GMLRs (b) CGMLRs (c) GMoE (d) CG-MoE (e) S-G-MoE (f) S-CG-MoE
  • Figure 4: Scatter plot of the contaminated tone perception data
  • Figure 5: Estimated mixing proportions for the first component of all the six fitted models on the contaminated tone perception data
  • ...and 1 more figures