A theoretical comparison of weight constraints in forecast combination and model averaging

TL;DR

The paper investigates how weight constraints shape the performance of forecast combination and model averaging. It theoretically and numerically compares four weight spaces across regression-, model-averaging-, performance-based, and eigenvector-style estimators, examining their effects on SSR, MSFE, bias, variance, and uniqueness. A Bayesian perspective and a conformal-inference-based numerical method are proposed to guide constraint choice, and simulation demonstrates how tighter constraints can reduce variance at the cost of bias, while looser constraints improve in-sample fit. The findings provide practical guidance for empirical researchers to select weight constraints based on prior information and forecasting targets, balancing interpretability, sparsity, and predictive accuracy. The study highlights that constraint choice is a critical, often overlooked, component of forecast-ensemble performance with tangible implications for real-world forecasting tasks.

Abstract

Forecast combination and model averaging have become popular tools in forecasting and prediction, both of which combine a set of candidate estimates with certain weights and are often shown to outperform single estimates. A data-driven method to determine combination/averaging weights typically optimizes a criterion under certain weight constraints. While a large number of studies have been devoted to developing and comparing various weight choice criteria, the role of weight constraints on the properties of combination forecasts is relatively less understood, and the use of various constraints in practice is also rather arbitrary. In this study, we summarize prevalent weight constraints used in the literature, and theoretically and numerically compare how they influence the properties of the combined forecast. Our findings not only provide a comprehensive understanding on the role of various weight constraints but also practical guidance for empirical researchers how to choose relevant constraints based on prior information and targets.

Figures (4)

• Figure 1: The schematic diagram for weight spaces.
• Figure 2: The schematic diagram for $\mathbb{W}^{\mathcal{E}}$.
• Figure 3: The schematic diagram for $\hat{{\bf w}}^{\mathcal{C}}_\mathrm{reg}$.
• Figure 4: The schematic diagram for $\hat{{\bf w}}^{\mathcal{D}}_\mathrm{reg}$.

Theorems & Definitions (7)

• Remark 1
• Remark 2
• Proposition 1
• Proposition 2
• Proposition 3
• Proposition 4
• Remark 3