Approximate Bayesian inference for cumulative probit regression models

Abstract

Ordinal categorical data are routinely encountered in many practical applications. When the primary goal is to construct a regression model for ordinal outcomes, cumulative link models represent one of the most popular choices to link the cumulative probabilities of the response with a set of covariates through a parsimonious linear predictor, shared across response categories. As the number of observations grows, standard sampling algorithms for Bayesian inference scale poorly, making posterior computation increasingly challenging for large datasets. In this article, we propose three scalable algorithms for approximating the posterior distribution of the regression coefficients in cumulative probit models relying on Variational Bayes and Expectation Propagation. We compare the proposed approaches with inference based on Markov Chain Monte Carlo, demonstrating superior computational performance and remarkable accuracy. Finally, we illustrate the utility of the proposed algorithms on a challenging case study to investigate the structure of a criminal network.

Figures (5)

• Figure 1: Construction of cumulative probit probabilities. The red curve represents the density of the latent $z_i$, distributed as a Gaussian with mean ${\bf x}_i {^\mathsf{ T}} {\boldsymbol \beta}$ and unit variance. Cumulative probabilities are obtained by discretizing such a distribution as in \ref{['eq:cum']}; colored regions correspond to the probabilities of distinct categories, defined in \ref{['eq:probs']}.
• Figure 2: Average absolute differences between posterior estimates obtained via mcmc and the proposed approximations. Values are displayed on log-scales for graphical clarity; boxplots represent variability across $100$ simulation replications.
• Figure 3: Computational times of the proposed approximations, in seconds (log scale); lines indicate the median across $100$ replications, while the shaded areas represent the first and third quartiles. Top panels report mcmc elapsed time, using a narrower axis scale for improved readability.
• Figure 4: Brazilian Bank. Univariate posterior densities estimated from mcmc samples (gray densities), ep (red curves), mfvb (orange curves) and pmf (blue curves). Accuracy scores for each method are reported in the top-left corners.
• Figure 5: Infinito network. Posterior distributions of the coefficients for the social-relation regression model; points denote posterior means, ticks $90\%$ credible intervals. Top panels: locale-specific and role-specific effects. Bottom panels: node-specific effects; red bars correspond to effects associated with bosses.