Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging Forecast Accuracy and Inventory KPIs: A Simulation-Based Software Framework

So Fukuhara, Abdallah Alabdallah, Nuwan Gunasekara, Slawomir Nowaczyk

TL;DR

This work addresses the gap between forecast accuracy and operational impact in spare-parts inventory for the automotive aftermarket by introducing a decision-centric simulation framework that links a synthetic demand generator, diverse forecasting models, and an inventory-cost simulator. The framework uses discrete-time simulation with a hierarchical Dealer–Truck–Part structure, survival-based demand, seasonality via Radial Basis Functions, and concept drift to generate realistic intermittent demand, enabling evaluation of forecasting methods through downstream KPIs like total cost and service level. Key findings show that improvements in MAE, RMSE, or $R^2$ do not consistently reduce inventory costs; classical intermittent-demand methods (Croston, SBA, TSB) can yield substantially lower costs than high-accuracy ML models in many scenarios, illustrating Simpson’s paradox in practice. The authors provide an open-source, reproducible testbed that guides model selection toward operational relevance, and suggest future work such as multi-echelon networks and richer cost structures to broaden applicability and impact.

Abstract

Efficient management of spare parts inventory is crucial in the automotive aftermarket, where demand is highly intermittent and uncertainty drives substantial cost and service risks. Forecasting is therefore central, but the quality of a forecasting model should be judged not by statistical accuracy (e.g., MAE, RMSE, IAE) but rather by its impact on key operational performance indicators (KPIs), such as total cost and service level. Yet most existing work evaluates models exclusively using accuracy metrics, and the relationship between these metrics and operational KPIs remains poorly understood. To address this gap, we propose a decision-centric simulation software framework that enables systematic evaluation of forecasting model in realistic inventory management setting. The framework comprises: (i) a synthetic demand generator tailored to spare-parts demand characteristics, (ii) a flexible forecasting module that can host arbitrary predictive models, and (iii) an inventory control simulator that consumes the forecasts and computes operational KPIs. This closed-loop setup enables researchers to evaluate models not only in terms of statistical error but also in terms of their downstream implications for inventory decisions. Using a wide range of simulation scenarios, we show that improvements in conventional accuracy metrics do not necessarily translate into better operational performance, and that models with similar statistical error profiles can induce markedly different cost-service trade-offs. We analyze these discrepancies to characterize how specific aspects of forecast performance affect inventory outcomes and derive guidance for model selection. Overall, the framework operationalizes the link between demand forecasting and inventory management, shifting evaluation from purely predictive accuracy toward operational relevance in the automotive aftermarket and related domains.

Bridging Forecast Accuracy and Inventory KPIs: A Simulation-Based Software Framework

TL;DR

This work addresses the gap between forecast accuracy and operational impact in spare-parts inventory for the automotive aftermarket by introducing a decision-centric simulation framework that links a synthetic demand generator, diverse forecasting models, and an inventory-cost simulator. The framework uses discrete-time simulation with a hierarchical Dealer–Truck–Part structure, survival-based demand, seasonality via Radial Basis Functions, and concept drift to generate realistic intermittent demand, enabling evaluation of forecasting methods through downstream KPIs like total cost and service level. Key findings show that improvements in MAE, RMSE, or do not consistently reduce inventory costs; classical intermittent-demand methods (Croston, SBA, TSB) can yield substantially lower costs than high-accuracy ML models in many scenarios, illustrating Simpson’s paradox in practice. The authors provide an open-source, reproducible testbed that guides model selection toward operational relevance, and suggest future work such as multi-echelon networks and richer cost structures to broaden applicability and impact.

Abstract

Efficient management of spare parts inventory is crucial in the automotive aftermarket, where demand is highly intermittent and uncertainty drives substantial cost and service risks. Forecasting is therefore central, but the quality of a forecasting model should be judged not by statistical accuracy (e.g., MAE, RMSE, IAE) but rather by its impact on key operational performance indicators (KPIs), such as total cost and service level. Yet most existing work evaluates models exclusively using accuracy metrics, and the relationship between these metrics and operational KPIs remains poorly understood. To address this gap, we propose a decision-centric simulation software framework that enables systematic evaluation of forecasting model in realistic inventory management setting. The framework comprises: (i) a synthetic demand generator tailored to spare-parts demand characteristics, (ii) a flexible forecasting module that can host arbitrary predictive models, and (iii) an inventory control simulator that consumes the forecasts and computes operational KPIs. This closed-loop setup enables researchers to evaluate models not only in terms of statistical error but also in terms of their downstream implications for inventory decisions. Using a wide range of simulation scenarios, we show that improvements in conventional accuracy metrics do not necessarily translate into better operational performance, and that models with similar statistical error profiles can induce markedly different cost-service trade-offs. We analyze these discrepancies to characterize how specific aspects of forecast performance affect inventory outcomes and derive guidance for model selection. Overall, the framework operationalizes the link between demand forecasting and inventory management, shifting evaluation from purely predictive accuracy toward operational relevance in the automotive aftermarket and related domains.
Paper Structure (12 sections, 9 equations, 6 figures, 1 table)

This paper contains 12 sections, 9 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Spare Parts Forecasting Methods
  3. Spare Parts Demand Data Generator
  4. Concept and Structure
  5. Survival Model
  6. Seasonality
  7. Concept Drift
  8. Simulation
  9. Input Data
  10. Cost Simulator
  11. Experiments
  12. Conclusions

Figures (6)

  • Figure 1: Class Diagram.
  • Figure 2: Curves of seasonality coefficients $w(t)$ for the period of 12 months (Jan-Dec): a) both seasons, b) summer, c) winter, d) no seasonality.
  • Figure 3: Overall flow of the software framework
  • Figure 4: ADI--CV$^{2}$ distribution for all demand series, with examples.
  • Figure 5: Comparison between total cost and each evaluation metric.
  • ...and 1 more figures