Variational Inference for Bayesian MIDAS Regression

TL;DR

A Coordinate Ascent Variational Inference algorithm for Bayesian Mixed Data Sampling (MIDAS) regression with linear weight parameterizations, creating a bilinear structure that renders generic Hamiltonian Monte Carlo samplers unreliable while preserving conditional conjugacy exploitable by CAVI.

Abstract

We develop a Coordinate Ascent Variational Inference (CAVI) algorithm for Bayesian Mixed Data Sampling (MIDAS) regression with linear weight parameterizations. The model separates impact coeffcients from weighting function parameters through a normalization constraint, creating a bilinear structure that renders generic Hamiltonian Monte Carlo samplers unreliable while preserving conditional conjugacy exploitable by CAVI. Each variational update admits a closed-form solution: Gaussian for regression coefficients and weight parameters, Inverse-Gamma for the error variance. The algorithm propagates uncertainty across blocks through second moments, distinguishing it from naive plug-in approximations. In a Monte Carlo study spanning 21 data-generating configurations with up to 50 predictors, CAVI produces posterior means nearly identical to a block Gibbs sampler benchmark while achieving speedups of 107x to 1,772x (Table 9). Generic automatic differentiation VI (ADVI), by contrast, produces bias 714 times larger while being orders of magnitude slower, confirming the value of model-specific derivations. Weight function parameters maintain excellent calibration (coverage above 92%) across all configurations. Impact coefficient credible intervals exhibit the underdispersion characteristic of mean-field approximations, with coverage declining from 89% to 55% as the number of predictors grows a documented trade-off between speed and interval calibration that structured variational methods can address. An empirical application to realized volatility forecasting on S&P 500 daily returns cofirms that CAVI and Gibbs sampling yield virtually identical point forecasts, with CAVI completing each monthly estimation in under 10 milliseconds.

Figures (11)

• Figure 1: Computational speedup of CAVI over Gibbs sampler as a function of the number of predictors $J$. Solid markers: measured. Open markers: extrapolated from Gibbs scaling pattern. Even at $J=50$, CAVI maintains a speedup of approximately $53\times$ (extrapolated).
• Figure 2: Gibbs sampler minimum effective sample size (ESS) as a function of $J$. The decline from 4,103 ($J=1$) to 539 ($J=10$) and further to 134 ($J=25$, 50 replications) indicates increasing autocorrelation in the MCMC chain as bilinear coupling involves more blocks.
• Figure 3: Empirical 95% coverage of active $\beta_j$ parameters as a function of sample size $T$, with $J=3$. Error bars show $\pm 1$ standard error across replications. Both methods improve with $T$. The CAVI--Gibbs gap narrows from approximately 10 percentage points at $T=100$ to 6 percentage points at $T=400$. The $T=50$ CAVI point (58.4%, Table \ref{['tab:samplesize']}) falls below the displayed range.
• Figure 4: Weight profile recovery for a representative replication ($J=3$, $T=200$). Left: decreasing profile ($\beta_1 = 2.0$). Center: hump-shaped profile ($\beta_2 = -1.0$). Right: decreasing profile ($\beta_3 = 0.5$). CAVI (blue) and Gibbs (red) estimates are nearly indistinguishable from the true profile (black) for strong predictors.
• Figure 5: S&P 500 monthly realized volatility (2000--2025). Top: raw $RV_t$. Bottom: $\log RV_t$. Dashed line: start of out-of-sample period. Prominent spikes correspond to the 2008 financial crisis and the 2020 COVID-19 shock.

Theorems & Definitions (7)

• Remark 1: Negative weights
• Remark 2: Choice of priors
• Remark 3: Failure of NUTS
• Proposition 1
• Proposition 2
• Proposition 3
• Proposition 4: Monotone convergence