Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Minimizing Spectral Risk Measures Applied to Markov Decision Processes

Nicole Bäuerle, Alexander Glauner

TL;DR

The paper develops a two-layer approach to minimizing spectral risk measures of total discounted costs in Markov Decision Processes with Borel spaces and potentially unbounded costs. By leveraging an infimum representation of spectral risk measures, it decomposes the problem into an inner MDP on an extended state space and an outer optimization over convex disutility functions, proving existence and enabling numerical approximation. The inner problem is solved via extended-state Bellman recursion for finite and infinite horizons, while the outer problem is shown to admit a solution and is approximated with piecewise-linear surrogates and convex conjugate techniques, with explicit error bounds. The framework is applied to a dynamic reinsurance problem, where stop-loss contracts emerge as optimal under common premium principles, illustrating the method’s practical relevance for risk-sensitive financial decisions.

Abstract

We study the minimization of a spectral risk measure of the total discounted cost generated by a Markov Decision Process (MDP) over a finite or infinite planning horizon. The MDP is assumed to have Borel state and action spaces and the cost function may be unbounded above. The optimization problem is split into two minimization problems using an infimum representation for spectral risk measures. We show that the inner minimization problem can be solved as an ordinary MDP on an extended state space and give sufficient conditions under which an optimal policy exists. Regarding the infinite dimensional outer minimization problem, we prove the existence of a solution and derive an algorithm for its numerical approximation. Our results include the findings in Bäuerle and Ott (2011) in the special case that the risk measure is Expected Shortfall. As an application, we present a dynamic extension of the classical static optimal reinsurance problem, where an insurance company minimizes its cost of capital.

Minimizing Spectral Risk Measures Applied to Markov Decision Processes

TL;DR

The paper develops a two-layer approach to minimizing spectral risk measures of total discounted costs in Markov Decision Processes with Borel spaces and potentially unbounded costs. By leveraging an infimum representation of spectral risk measures, it decomposes the problem into an inner MDP on an extended state space and an outer optimization over convex disutility functions, proving existence and enabling numerical approximation. The inner problem is solved via extended-state Bellman recursion for finite and infinite horizons, while the outer problem is shown to admit a solution and is approximated with piecewise-linear surrogates and convex conjugate techniques, with explicit error bounds. The framework is applied to a dynamic reinsurance problem, where stop-loss contracts emerge as optimal under common premium principles, illustrating the method’s practical relevance for risk-sensitive financial decisions.

Abstract

We study the minimization of a spectral risk measure of the total discounted cost generated by a Markov Decision Process (MDP) over a finite or infinite planning horizon. The MDP is assumed to have Borel state and action spaces and the cost function may be unbounded above. The optimization problem is split into two minimization problems using an infimum representation for spectral risk measures. We show that the inner minimization problem can be solved as an ordinary MDP on an extended state space and give sufficient conditions under which an optimal policy exists. Regarding the infinite dimensional outer minimization problem, we prove the existence of a solution and derive an algorithm for its numerical approximation. Our results include the findings in Bäuerle and Ott (2011) in the special case that the risk measure is Expected Shortfall. As an application, we present a dynamic extension of the classical static optimal reinsurance problem, where an insurance company minimizes its cost of capital.

Paper Structure

This paper contains 9 sections, 16 theorems, 99 equations.

Table of Contents

  1. Introduction
  2. Spectral Risk Measures
  3. Markov Decision Model
  4. Inner Problem: Finite Planning Horizon
  5. Inner Problem: Infinite Planning Horizon
  6. Relaxed Assumptions for Monotone Models
  7. Outer Problem: Existence
  8. Outer Problem: Numerical Approximation
  9. Dynamic Optimal Reinsurance

Key Result

Proposition 2.5

Let $\rho_{\phi}$ be a spectral risk measure. Then $\mu$ is a probability measure on $[0,1]$ and $\rho_{\phi}$ has the representation

Theorems & Definitions (41)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Example 2.4
  • Proposition 2.5
  • Proposition 2.6
  • proof
  • Remark 2.7
  • Definition 3.2
  • Proposition 4.1
  • ...and 31 more