Temporal Difference Learning for High-Dimensional PIDEs with Jumps

TL;DR

The paper addresses solving high-dimensional PIDEs with jumps, which are challenging due to non-local terms and the curse of dimensionality. It develops a Lévy-type forward-backward SDE framework and trains a two-output neural network to approximate both the solution $u(t,x)$ and the non-local integral term, using a temporal-difference loss augmented with termination and martingale constraints. The approach achieves $O(10^{-4})$ accuracy in 1D and $O(10^{-3})$ accuracy in 100D with moderate numbers of trajectories and shows robustness to jump form and intensity, as well as scalable, near-linear runtime growth in dimension. This provides a practical, scalable method for high-dimensional PIDEs in finance and related fields where jump processes are essential.

Abstract

In this paper, we propose a deep learning framework for solving high-dimensional partial integro-differential equations (PIDEs) based on the temporal difference learning. We introduce a set of Levy processes and construct a corresponding reinforcement learning model. To simulate the entire process, we use deep neural networks to represent the solutions and non-local terms of the equations. Subsequently, we train the networks using the temporal difference error, termination condition, and properties of the non-local terms as the loss function. The relative error of the method reaches O(10^{-3}) in 100-dimensional experiments and O(10^{-4}) in one-dimensional pure jump problems. Additionally, our method demonstrates the advantages of low computational cost and robustness, making it well-suited for addressing problems with different forms and intensities of jumps.

Figures (8)

• Figure 1: Architecture of the residual network utilized in this work. The neural network takes a $d+1$-dimensional input consisting of time $t\in\mathbb{R}$ and spatial variable $x\in\mathbb{R}^d$. The two outputs represent $u(t,x)$ and $\int_{\mathbb{R}^d}(u(t,x+G(x,z)) - u(t,x))\nu(\mathop{}\!\mathrm{d} z)$ respectively.
• Figure 1: Simulation of jumps. To simulate the jumps on each trajectory, the following steps are taken: (a) A sequence of exponential distributions with parameter $\lambda$ is generated. (b) The cumulative sum of this sequence yields the arrival times of the Poisson process on the respective trajectory. (c) At each arrival time, a single sample is drawn from the distribution $\phi(z)$ to determine the value of the jump at that particular moment.
• Figure 2: The diagram of solving high-dimensional PIDEs with jumps by temporal difference method. The PIDE under consideration is associated with a set of Lévy-type forward-backward stochastic processes. These processes can be effectively characterized and addressed within the framework of reinforcement learning, as depicted in the highlighted red box in the figure. By employing neural networks, the calculation of the loss function becomes feasible through the utilization of temporal difference methods. Consequently, the model parameters can be updated accordingly to facilitate the resolution of the PIDE.
• Figure 2: Relative error of one-dimensional pure jump problem. (a) The evolution of the relative error of $Y_0$ with respect to the number of iterations. The exact value is $Y_0=1$, and the relative error converges to 0.02%. (b) The relative error of the neural network's approximation for $Y_t$ at different time $t$.
• Figure 3: The visualization of the trajectories for the one-dimensional pure jump problem. 30 trajectories are displayed, and 5 trajectories exhibit jumps, while the remaining 25 trajectories coincide entirely due to the absence of Brownian motion.

Theorems & Definitions (1)

• Remark 2.1