Algorithmic Transparency in Forecasting Support Systems

TL;DR

It is argued that algorithmic transparency may be a key factor towards better, integrative forecasting and test this assertion with three FSS designs that vary in their degrees of transparency based on time series decomposition.

Abstract

Most organizations adjust their statistical forecasts (e.g. on sales) manually. Forecasting Support Systems (FSS) enable the related process of automated forecast generation and manual adjustments. As the FSS user interface connects user and statistical algorithm, it is an obvious lever for facilitating beneficial adjustments whilst discouraging harmful adjustments. This paper reviews and organizes the literature on judgemental forecasting, forecast adjustments, and FSS design. I argue that algorithmic transparency may be a key factor towards better, integrative forecasting and test this assertion with three FSS designs that vary in their degrees of transparency based on time series decomposition. I find transparency to reduce the variance and amount of harmful forecast adjustments. Letting users adjust the algorithm's transparent components themselves, however, leads to widely varied and overall most detrimental adjustments. Responses indicate a risk of overwhelming users with algorithmic transparency without adequate training. Accordingly, self-reported satisfaction is highest with a non-transparent FSS.

Figures (9)

• Figure 1: Transparently Adjustable FSS design in its final iteration.
• Figure 2: TA FSS: Detail view of weekly effects which allows for adjustments by dragging the white handle points.
• Figure 3: $AV$ and $rMAE$ per FSS design conditional on positive $AV$. White squares mark the mean values.
• Figure 4: Counts of voluntary free-text comments.
• Figure 5: $Adjustment Volume$ and proportion of $good$ adjustments, marked by each bar's lower part, per product and FSS design. Values under the product index indicate Prophet's $rMAE$ relative to a simple exponential smoothing model.