The Nonstationarity-Complexity Tradeoff in Return Prediction

TL;DR

The paper tackles non-stationarity in stock return prediction by jointly optimizing model class and training window size, revealing a fundamental nonstationarity-complexity tradeoff where more complex models require longer estimation windows that invite regime shifts. It introduces ATOMS, an adaptive tournament model selection framework that adaptively validates candidates on non-stationary data and provides theoretical guarantees for near-optimal performance, including an $R^2$-oriented variant. Empirically, it shows substantial out-of-sample improvements across 17 industry portfolios (average $R^2$ gains of 14–23% over fixed-horizon baselines) and particularly strong results during recessions (Gulf War, 2001, 2008), with a trading strategy delivering about 31% higher cumulative wealth on average. The approach integrates a rich predictor set (CPZ24 factors, FF3, GKX characteristics, and lagged returns) and demonstrates cross-industry robustness, suggesting practical value for adaptive asset-pricing and investment decisions in non-stationary markets.

Abstract

We investigate machine learning models for stock return prediction in non-stationary environments, revealing a fundamental nonstationarity-complexity tradeoff: complex models reduce misspecification error but require longer training windows that introduce stronger non-stationarity. We resolve this tension with a novel model selection method that jointly optimizes model class and training window size using a tournament procedure that adaptively evaluates candidates on non-stationary validation data. Our theoretical analysis demonstrates that this approach balances misspecification error, estimation variance, and non-stationarity, performing close to the best model in hindsight. Applying our method to 17 industry portfolio returns, we consistently outperform standard rolling-window benchmarks, improving out-of-sample $R^2$ by 14-23% on average. During NBER-designated recessions, improvements are substantial: our method achieves positive $R^2$ during the Gulf War recession while benchmarks are negative, and improves $R^2$ in absolute terms by at least 80bps during the 2001 recession as well as superior performance during the 2008 Financial Crisis. Economically, a trading strategy based on our selected model generates 31% higher cumulative returns averaged across the industries.

Figures (15)

• Figure 1: Number of Industries where Each Model Attains the Highest Annual Out-of-Sample $R^2$.
• Figure 2: Annual Out-of-Sample $R^2$ of Three Models for $17$ Industry Portfolios.
• Figure 3: Our Framework for Model Training and Selection under Non-stationarity.
• Figure 4: Box Plot of Out-of-Sample $R^2$ of $\texttt{ATOMS}$ and Fixed-Window Baselines for $17$ Industry Portfolios.
• Figure 5: Annual Out-of-Sample $R^2$ of $\texttt{ATOMS}$ and Fixed-Window Baselines for $17$ Industry Portfolios.

Theorems & Definitions (43)

• Example 3.1: Finite class
• Example 3.2: Linear class
• Example 3.3: Kernel class
• Theorem 3.1: Prediction error bound
• proof : Proof of \ref{['sm:thm-tradeoff']}
• Example 3.4: Selection of model class and window under non-stationarity
• Lemma 4.1: Computational complexity
• proof : Proof of \ref{['sm:lem-complexity-tournament']}
• Theorem 4.1: Near-optimal model comparison
• proof : Proof of \ref{['sm:thm-model-comparison']}