Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Order Optimal Regret Bounds for Sharpe Ratio Optimization under Thompson Sampling

Mohammad Taha Shah, Sabrina Khurshid, Gourab Ghatak

TL;DR

Theoretical contributions include a novel regret decomposition specifically designed for the Sharpe ratio, highlighting the role of information acquisition about the reward distribution in driving learning efficiency and establishing fundamental performance limits for the proposed algorithm in terms of an upper bound on regret.

Abstract

In this paper, we investigate the problem of sequential decision-making for Sharpe ratio (SR) maximization in a stochastic bandit setting. We focus on the Thompson Sampling (TS) algorithm, a Bayesian approach celebrated for its empirical performance and exploration efficiency, under the assumption of Gaussian rewards with unknown parameters. Unlike conventional bandit objectives focusing on maximizing cumulative reward, Sharpe ratio optimization instead introduces an inherent tradeoff between achieving high returns and controlling risk, demanding careful exploration of both mean and variance. Our theoretical contributions include a novel regret decomposition specifically designed for the Sharpe ratio, highlighting the role of information acquisition about the reward distribution in driving learning efficiency. Then, we establish fundamental performance limits for the proposed algorithm \texttt{SRTS} in terms of an upper bound on regret. We also derive the matching lower bound and show the order-optimality. Our results show that Thompson Sampling achieves logarithmic regret over time, with distribution-dependent factors capturing the difficulty of distinguishing arms based on risk-adjusted performance. Empirical simulations show that our algorithm significantly outperforms existing algorithms.

Order Optimal Regret Bounds for Sharpe Ratio Optimization under Thompson Sampling

TL;DR

Theoretical contributions include a novel regret decomposition specifically designed for the Sharpe ratio, highlighting the role of information acquisition about the reward distribution in driving learning efficiency and establishing fundamental performance limits for the proposed algorithm in terms of an upper bound on regret.

Abstract

In this paper, we investigate the problem of sequential decision-making for Sharpe ratio (SR) maximization in a stochastic bandit setting. We focus on the Thompson Sampling (TS) algorithm, a Bayesian approach celebrated for its empirical performance and exploration efficiency, under the assumption of Gaussian rewards with unknown parameters. Unlike conventional bandit objectives focusing on maximizing cumulative reward, Sharpe ratio optimization instead introduces an inherent tradeoff between achieving high returns and controlling risk, demanding careful exploration of both mean and variance. Our theoretical contributions include a novel regret decomposition specifically designed for the Sharpe ratio, highlighting the role of information acquisition about the reward distribution in driving learning efficiency. Then, we establish fundamental performance limits for the proposed algorithm \texttt{SRTS} in terms of an upper bound on regret. We also derive the matching lower bound and show the order-optimality. Our results show that Thompson Sampling achieves logarithmic regret over time, with distribution-dependent factors capturing the difficulty of distinguishing arms based on risk-adjusted performance. Empirical simulations show that our algorithm significantly outperforms existing algorithms.

Paper Structure

This paper contains 46 sections, 16 theorems, 151 equations, 2 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Risk-aware bandits
  4. Thompson sampling and Sharpe ratio
  5. Contributions
  6. Problem Formulation
  7. The Sequential Decision-Making Environment
  8. The Risk-Aware Objective: The Sharpe Ratio
  9. Algorithmic Reward and Regret
  10. Bounding Pull-Count Variance
  11. Notations
  12. Regret Decomposition
  13. Proof Sketch of Theorem \ref{['th:th_dec']}
  14. SRTS algorithm for the Gaussian distribution
  15. The Bayesian Model and Posterior Updates
  16. ...and 31 more sections

Key Result

lemma 1

For $s_{i,n} = \sum_{t=1}^n \mathbb{I}\!\left(\pi(t) = i\right)$, the variance of $s_{i,n}$ satisfies

Figures (2)

  • Figure 1: The hierarchical dependence structure of the SRTS generative process. The non-linear fractional formulation of the SR intrinsically couples the sub-Gaussian mean sample with the heavy-tailed Gamma precision sample, bypassing standard MAB concentration bounds.
  • Figure 2: (a) Here we have three results: (1) Regret when $\rho = 1$, (2) Regret when $\rho = 0$, and (3) Regret when $\rho = 1$ and $\mu_i = 1$, $i \in \{1,2,\dots,K\}$. (b) Performance of SRTS w.r.t UCB-RSSR and U-UCB for $\rho = 1$. (c) Regret v/s $\rho$ for Gaussian SRTS w.r.t UCB-RSSR and U-UCB.

Theorems & Definitions (44)

  • Definition 1
  • Definition 2
  • lemma 1
  • proof
  • Theorem 1
  • Remark 1
  • Theorem 2
  • Theorem 3
  • Remark 2
  • Remark 3
  • ...and 34 more