RegimeFolio: A Regime Aware ML System for Sectoral Portfolio Optimization in Dynamic Markets

TL;DR

This paper tackles non-stationary financial markets by introducing RegimeFolio, a regime-aware, sector-specialized portfolio framework that explicitly segments volatility regimes via a VIX-based classifier, trains sector-specific ensemble forecasters within each regime, and performs regime-conditioned mean-variance optimization with shrinkage covariance. By aligning both forecasts and allocations to the prevailing market state, RegimeFolio achieves robust performance across volatility regimes, outperforming regime-agnostic ML baselines and the S&P 500 on 34 large-cap U.S. equities from 2020–2024. The approach yields up to 15–20% improvements in forecast accuracy and substantively higher risk-adjusted returns, with a cumulative return of 137.0% and a Sharpe ratio of 1.17, while reducing maximum drawdown. The framework emphasizes interpretability, scalability, and deployment readiness, offering a practical blueprint for regime-sensitive, sector-aware portfolio management with potential extensions to multi-asset universes and live trading.

Abstract

Financial markets are inherently non-stationary, with shifting volatility regimes that alter asset co-movements and return distributions. Standard portfolio optimization methods, typically built on stationarity or regime-agnostic assumptions, struggle to adapt to such changes. To address these challenges, we propose RegimeFolio, a novel regime-aware and sector-specialized framework that, unlike existing regime-agnostic models such as DeepVol and DRL optimizers, integrates explicit volatility regime segmentation with sector-specific ensemble forecasting and adaptive mean-variance allocation. This modular architecture ensures forecasts and portfolio decisions remain aligned with current market conditions, enhancing robustness and interpretability in dynamic markets. RegimeFolio combines three components: (i) an interpretable VIX-based classifier for market regime detection; (ii) regime and sector-specific ensemble learners (Random Forest, Gradient Boosting) to capture conditional return structures; and (iii) a dynamic mean-variance optimizer with shrinkage-regularized covariance estimates for regime-aware allocation. We evaluate RegimeFolio on 34 large cap U.S. equities from 2020 to 2024. The framework achieves a cumulative return of 137 percent, a Sharpe ratio of 1.17, a 12 percent lower maximum drawdown, and a 15 to 20 percent improvement in forecast accuracy compared to conventional and advanced machine learning benchmarks. These results show that explicitly modeling volatility regimes in predictive learning and portfolio allocation enhances robustness and leads to more dependable decision-making in real markets.

Figures (5)

• Figure 1: Systematic Methodology Framework for Regime Aware Cross-Sectoral Portfolio Optimization
• Figure 2: Feature Importance Heatmap: Random Forest and Gradient Boosting Across Volatility Regimes. The heatmaps display the relative importance of features (rows) across low, medium, and high volatility regimes (columns). Warmer colors (orange-red) indicate higher feature importance, while cooler colors (blue) represent lower importance. Values are normalized, with those closer to 1 indicating stronger predictive power for that feature in the corresponding volatility regime.
• Figure 3: SHAP summary plots for three different regimes, showing feature importance and impact on model predictions.
• Figure 4: Comprehensive Performance Analysis. Panel (a) presents the annualized returns of the proposed RegimeFolio (Regime-Aware Strategy) across varying volatility regimes, demonstrating its adaptability and superior performance relative to the Equal-Weight Portfolio and the Russell 2000 Index. Panel (b) shows sector-wise performance from 2020 to 2024, where RegimeFolio (Regime-Aware Strategy) consistently outperforms alternative approaches across all major economic sectors.
• Figure 5: Superior Portfolio Performance: RegimeFolio (Regime-Aware Strategy) vs S&P 500 Benchmark (2020–2024). The RegimeFolio approach demonstrates 63.2% excess returns, a higher Sharpe ratio (1.17 vs 0.66), and improved risk control. Shaded regions indicate volatility regimes (VIX terciles).