Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MARS: A Meta-Adaptive Reinforcement Learning Framework for Risk-Aware Multi-Agent Portfolio Management

Jiayi Chen, Jing Li, Guiling Wang

TL;DR

MARS tackles non-stationarity and risk in portfolio management by combining a heterogeneous ensemble of Safety-Critic agents with explicit risk profiles and a high level Meta-Adaptive Controller that dynamically orchestrates the ensemble. The two-tier design enables robust adaptation across market regimes while enforcing practical trading constraints via a risk overlay. Empirical results on DJI and HSI show superior risk-adjusted returns and capital preservation in bear markets, with ablations confirming the necessity of both MAC and HAE components for performance gains. The work offers a scalable, adaptive framework for risk-aware multi-agent portfolio optimization with potential for real-world deployment.

Abstract

Reinforcement Learning (RL) has shown significant promise in automated portfolio management; however, effectively balancing risk and return remains a central challenge, as many models fail to adapt to dynamically changing market conditions. We propose Meta-controlled Agents for a Risk-aware System (MARS), a novel framework addressing this through a multi-agent, risk-aware approach. MARS replaces monolithic models with a Heterogeneous Agent Ensemble, where each agent's unique risk profile is enforced by a Safety-Critic network to span behaviors from capital preservation to aggressive growth. A high-level Meta-Adaptive Controller (MAC) dynamically orchestrates this ensemble, shifting reliance between conservative and aggressive agents to minimize drawdown during downturns while seizing opportunities in bull markets. This two-tiered structure leverages behavioral diversity rather than explicit feature engineering to ensure a disciplined portfolio robust across market regimes. Experiments on major international indexes confirm that our framework significantly reduces maximum drawdown and volatility while maintaining competitive returns.

MARS: A Meta-Adaptive Reinforcement Learning Framework for Risk-Aware Multi-Agent Portfolio Management

TL;DR

MARS tackles non-stationarity and risk in portfolio management by combining a heterogeneous ensemble of Safety-Critic agents with explicit risk profiles and a high level Meta-Adaptive Controller that dynamically orchestrates the ensemble. The two-tier design enables robust adaptation across market regimes while enforcing practical trading constraints via a risk overlay. Empirical results on DJI and HSI show superior risk-adjusted returns and capital preservation in bear markets, with ablations confirming the necessity of both MAC and HAE components for performance gains. The work offers a scalable, adaptive framework for risk-aware multi-agent portfolio optimization with potential for real-world deployment.

Abstract

Reinforcement Learning (RL) has shown significant promise in automated portfolio management; however, effectively balancing risk and return remains a central challenge, as many models fail to adapt to dynamically changing market conditions. We propose Meta-controlled Agents for a Risk-aware System (MARS), a novel framework addressing this through a multi-agent, risk-aware approach. MARS replaces monolithic models with a Heterogeneous Agent Ensemble, where each agent's unique risk profile is enforced by a Safety-Critic network to span behaviors from capital preservation to aggressive growth. A high-level Meta-Adaptive Controller (MAC) dynamically orchestrates this ensemble, shifting reliance between conservative and aggressive agents to minimize drawdown during downturns while seizing opportunities in bull markets. This two-tiered structure leverages behavioral diversity rather than explicit feature engineering to ensure a disciplined portfolio robust across market regimes. Experiments on major international indexes confirm that our framework significantly reduces maximum drawdown and volatility while maintaining competitive returns.

Paper Structure

This paper contains 29 sections, 9 equations, 5 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem Formulation
  5. Overall Training Algorithm
  6. Heterogeneous Agent Ensemble (HAE)
  7. Actor Network $\pi_{\phi_i}(s_t)$
  8. Critic Network $Q_{\psi_i}(s_t, a_t)$
  9. Safety-Critic Network $C_{\xi_i}(s_t, a_t)$
  10. Meta-Adaptive Controller (MAC)
  11. Trading Procedure
  12. Experiments
  13. Experimental Setup
  14. Datasets
  15. Evaluation Metrics
  16. ...and 14 more sections

Figures (5)

  • Figure 1: The MARS framework architecture. The system processes the Market State ($s_t$) through two parallel components. The Meta-Adaptive Controller (MAC) produces agent weights ($w_t$), while the Heterogeneous Agent Ensemble (HAE) generates proposed actions ($a_t^i$). These outputs are aggregated and passed through a Risk Management Overlay to produce the final executed action ($A'_t$).
  • Figure 2: Performance comparison on the DJI 2022 dataset. MARS (red) shows superior capital preservation with a significantly shallower drawdown compared to baselines.
  • Figure 3: Performance comparison on the DJI 2024 dataset. MARS (red) achieves the highest return while maintaining a competitive drawdown profile.
  • Figure 4: Ablation study performance on the DJI 2024 dataset. The main MARS model (red) outperforms all variants, validating its architectural components.
  • Figure 5: Comparison of agent allocation strategies under Meta-Adaptive Controller (MAC) during the 2022 bear market (top) and the 2024 bull market (bottom) for DJI portfolio.