Stochastic Optimization Algorithms for Instrumental Variable Regression with Streaming Data
Xuxing Chen, Abhishek Roy, Yifan Hu, Krishnakumar Balasubramanian
TL;DR
This work addresses instrumental variable regression with streaming data by recasting IVaR as a conditional stochastic optimization and developing fully online algorithms that avoid matrix inversions and mini-batches. It introduces two algorithms based on oracle access: TOSG-IVaR uses a two-sample gradient estimator and achieves last-iterate convergence with rate $\mathcal{O}(\log T / T)$ for linear models, while OTSG-IVaR handles the one-sample streaming setting with $\mathcal{O}(1/T^{1-\iota})$ convergence for any $\iota>0$ under mild assumptions. The methods avoid explicit modeling of the $Z$–$X$ relationship and do not rely on nested sampling or minimax dual formulations, yielding memory-efficient online IV regression. Empirical results corroborate theory, showing robust performance across dimensions and outperforming standard online 2SLS baselines, with clear advantages in per-iteration memory and stability. These contributions enable scalable IV regression in streaming contexts and open avenues for online inference and extensions to nonlinear or nonparametric IVaR.
Abstract
We develop and analyze algorithms for instrumental variable regression by viewing the problem as a conditional stochastic optimization problem. In the context of least-squares instrumental variable regression, our algorithms neither require matrix inversions nor mini-batches and provides a fully online approach for performing instrumental variable regression with streaming data. When the true model is linear, we derive rates of convergence in expectation, that are of order $\mathcal{O}(\log T/T)$ and $\mathcal{O}(1/T^{1-ι})$ for any $ι>0$, respectively under the availability of two-sample and one-sample oracles, respectively, where $T$ is the number of iterations. Importantly, under the availability of the two-sample oracle, our procedure avoids explicitly modeling and estimating the relationship between confounder and the instrumental variables, demonstrating the benefit of the proposed approach over recent works based on reformulating the problem as minimax optimization problems. Numerical experiments are provided to corroborate the theoretical results.