Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Learning for Constrained Utility Maximisation

Ashley Davey, Harry Zheng

TL;DR

This paper proposes two algorithms for solving stochastic control problems with deep learning, with a focus on the utility maximisation problem, and uses the full power of the duality method to solve non-Markovian problems, which are often beyond the scope of stoChastic control solvers in the existing literature.

Abstract

This paper proposes two algorithms for solving stochastic control problems with deep learning, with a focus on the utility maximisation problem. The first algorithm solves Markovian problems via the Hamilton Jacobi Bellman (HJB) equation. We solve this highly nonlinear partial differential equation (PDE) with a second order backward stochastic differential equation (2BSDE) formulation. The convex structure of the problem allows us to describe a dual problem that can either verify the original primal approach or bypass some of the complexity. The second algorithm utilises the full power of the duality method to solve non-Markovian problems, which are often beyond the scope of stochastic control solvers in the existing literature. We solve an adjoint BSDE that satisfies the dual optimality conditions. We apply these algorithms to problems with power, log and non-HARA utilities in the Black-Scholes, the Heston stochastic volatility, and path dependent volatility models. Numerical experiments show highly accurate results with low computational cost, supporting our proposed algorithms.

Deep Learning for Constrained Utility Maximisation

TL;DR

This paper proposes two algorithms for solving stochastic control problems with deep learning, with a focus on the utility maximisation problem, and uses the full power of the duality method to solve non-Markovian problems, which are often beyond the scope of stoChastic control solvers in the existing literature.

Abstract

This paper proposes two algorithms for solving stochastic control problems with deep learning, with a focus on the utility maximisation problem. The first algorithm solves Markovian problems via the Hamilton Jacobi Bellman (HJB) equation. We solve this highly nonlinear partial differential equation (PDE) with a second order backward stochastic differential equation (2BSDE) formulation. The convex structure of the problem allows us to describe a dual problem that can either verify the original primal approach or bypass some of the complexity. The second algorithm utilises the full power of the duality method to solve non-Markovian problems, which are often beyond the scope of stochastic control solvers in the existing literature. We solve an adjoint BSDE that satisfies the dual optimality conditions. We apply these algorithms to problems with power, log and non-HARA utilities in the Black-Scholes, the Heston stochastic volatility, and path dependent volatility models. Numerical experiments show highly accurate results with low computational cost, supporting our proposed algorithms.

Paper Structure

This paper contains 15 sections, 3 theorems, 63 equations, 6 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. The Utility Maximisation Problem
  3. The Deep Controlled 2BSDE Method for the Markovian Case
  4. The Deep Controlled 2BSDE Method
  5. Extension of the Algorithm to the Dual Problem
  6. Numerical Examples
  7. The Deep SMP Method for the Non-Markovian Case
  8. The Deep SMP Method
  9. Numerical Examples
  10. Methodology Comparison
  11. Parameter Comparison
  12. Comparison to Literature
  13. Conclusion
  14. Proof of Theorem \ref{['2BSDE_thm']}
  15. Algorithm

Key Result

Theorem 3.2

Suppose that $u \in \mathcal{C}^{1 ,3}([0,T) \times \mathbb{R}^d) \cap \mathcal{C}^0([0,T] \times \mathbb{R}^d)$, and there exists an optimal control $\alpha \in {\cal A}$ with associated controlled diffusion $X$, defined by (update). Then there exist continuous processes $(V, Z, \Gamma)$, valued in with the terminal conditions $V(T) = g(\mathcal{X}(T))$ and $Z(T) = D_xg(\mathcal{X}(T))$, where

Figures (6)

  • Figure 1: Loss functions against iteration step for the primal and dual deep controlled 2BSDE methods applied to the unconstrained non-HARA utility problem.
  • Figure 2: Two simulations of the value process and value-derivative process for the primal and dual deep controlled 2BSDE methods applied to the unconstrained non-HARA utility problem. The displayed dual processes are those implied by the duality relations (\ref{['utility_relations']}).
  • Figure 3: Approximation and relative error of the value function against number of time steps $N$ for the primal and dual deep controlled 2BSDE methods applied to the unconstrained non-HARA utility problem. Note in the left graph that the Y values on the axis are shifted down by 2.253.
  • Figure 4: Approximation of the value function and bsde loss against iteration step for the primal and dual deep controlled 2BSDE methods applied to the cone constrained power utility problem.
  • Figure 5: Accuracy and runtime of the primal deep controlled 2BSDE method applied to unconstrained power utility problems with different dimensions.
  • ...and 1 more figures

Theorems & Definitions (19)

  • Remark 3.1
  • Theorem 3.2
  • Remark 3.3
  • Remark 3.4
  • Remark 3.5
  • Remark 3.6
  • Theorem 3.7: li2018dynamic, Theorem 12
  • Example 3.8: Unconstrained Non-HARA Utility Problem
  • Example 3.9: Cone-Constrained Merton Problem
  • Remark 3.10
  • ...and 9 more