Numerical Discretization Methods for the Extended Linear Quadratic Control Problem

TL;DR

This work develops and analyzes three numerical discretization strategies—ODE-based, matrix-exponential, and step-doubling—for transforming continuous-time LQ-OCPs into accurate discrete-time equivalents, applicable to both deterministic and stochastic settings. By formulating differential-equation systems that capture the evolution of discretization-compatible matrices ($A,B,R_{ww},Q,M$) and by extending to stochastic costs via Euler–Maruyama discretization, the authors show that the discrete problems faithfully reflect the original control objectives. The matrix-exponential approach provides exact reformulations, while the step-doubling method offers a faster alternative with equivalent accuracy, and the stochastic-cost analysis reveals a generalized $ ext{χ}^2$ distribution for the reformulated costs. Experimental results confirm numerical equivalence and highlight the step-doubling method as the fastest option, enabling efficient and reliable deployment of discretized LQ-OCPs in digital environments. This work thus advances practical, provably equivalent discrete-time representations for extended LQ-OCPs in both deterministic and stochastic contexts.

Abstract

In this study, we introduce numerical methods for discretizing continuous-time linear-quadratic optimal control problems (LQ-OCPs). The discretization of continuous-time LQ-OCPs is formulated into differential equation systems, and we can obtain the discrete equivalent by solving these systems. We present the ordinary differential equation (ODE), matrix exponential, and a novel step-doubling method for the discretization of LQ-OCPs. Utilizing Euler-Maruyama discretization with a fine step, we reformulate the costs of continuous-time stochastic LQ-OCPs into a quadratic form, and show that the stochastic cost follows the $χ^2$ distribution. In the numerical experiment, we test and compare the proposed numerical methods. The results ensure that the discrete-time LQ-OCP derived using the proposed numerical methods is equivalent to the original problem.

Figures (3)

• Figure 1: The exponential of a matrix $A_c$. The blue line is the true result computed with expm() in MATLAB. The red dots are the results of the step-doubling method.
• Figure 2: The error and CPU time of the ODE methods and the step-doubling methods with different discretization methods. The error is $e(i) = |i(T_s) - i(N)|$ for $i \in [A, B, R_{ww}, M, Q]$, where $i(T_s)$ is the true result from the matrix exponential method.
• Figure 3: The likelihood of the cost functions of continuous-time and discrete-time stochastic LQ-OCPs with 30000 Monte Carlo simulations. The Cont., Disc. EM. indicate the continuous-time, discrete-time, and EM reformulated stochastic costs, respectively. $E\{\phi\}$-Analy. is the analytic expectation described by \ref{['eq:analyticMeanOfStochasticCosts']}, where the continuous-time element $\text{tr}(Q_{c,ww}P_w)$ is solved using the EM method with $N=2^8$.

Theorems & Definitions (5)

• Remark 1
• Proposition 1: Discretization of the deterministic LQ-OCP
• Proposition 2: Discretization of the linear SDE
• Proposition 3: Discretization of the stochastic LQ-OCP
• Proposition 4: Distribution of the stochastic costs