Reinforcement Learning for Monetary Policy Under Macroeconomic Uncertainty: Analyzing Tabular and Function Approximation Methods

TL;DR

The paper evaluates nine reinforcement learning methods for dynamic monetary policy under macroeconomic uncertainty, using a data-driven, FRED-based environment with a linear-Gaussian state transition and a discrete-action MDP. Contrary to expectations, simple tabular Q-learning often outperforms deep and Bayesian methods and traditional rules, highlighting robustness of discretized policies in this setting. The results underscore both the potential and current limitations of applying modern RL to macro policy, suggesting RL as a decision-support tool with uncertainty quantification rather than a replacement for human judgment. The work also emphasizes the importance of problem characteristics, discretization, and stability when selecting RL algorithms for economic applications.

Abstract

We study how a central bank should dynamically set short-term nominal interest rates to stabilize inflation and unemployment when macroeconomic relationships are uncertain and time-varying. We model monetary policy as a sequential decision-making problem where the central bank observes macroeconomic conditions quarterly and chooses interest rate adjustments. Using publically accessible historical Federal Reserve Economic Data (FRED), we construct a linear-Gaussian transition model and implement a discrete-action Markov Decision Process with a quadratic loss reward function. We chose to compare nine different reinforcement learning style approaches against Taylor Rule and naive baselines, including tabular Q-learning variants, SARSA, Actor-Critic, Deep Q-Networks, Bayesian Q-learning with uncertainty quantification, and POMDP formulations with partial observability. Surprisingly, standard tabular Q-learning achieved the best performance (-615.13 +- 309.58 mean return), outperforming both enhanced RL methods and traditional policy rules. Our results suggest that while sophisticated RL techniques show promise for monetary policy applications, simpler approaches may be more robust in this domain, highlighting important challenges in applying modern RL to macroeconomic policy.

Figures (5)

• Figure 1: Performance comparison across all methods showing discounted returns, inflation loss, unemployment loss, and total loss per step. Box plots display median, quartiles, and outliers for each method across evaluation episodes.
• Figure 2: Learning dynamics dashboard showing training curves, epsilon decay, loss evolution, uncertainty evolution, convergence analysis, training stability, performance distributions, and sample efficiency across all methods.
• Figure 3: Statistical analysis dashboard showing performance distribution comparisons, return histograms, statistical significance matrix with p-values, risk-return scatter plot, correlation analysis, uncertainty comparisons, and multi-criteria method rankings.
• Figure 4: Graphs showing policy decisions in inflation-unemployment space, policy rate matrices, Phillips curve trade-offs, action interpretations, economic loss components, policy reactivity to shocks, Taylor rule adherence, and performance across different economic scenarios.
• Figure 5: Policy action analysis showing action distribution heatmaps, method-specific action preferences, Q-value distributions, and policy decisiveness measures across all implemented approaches.