Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Controlled Diffusions under Full, Partial and Decentralized Information: Existence of Optimal Policies and Discrete-Time Approximations

Somnath Pradhan, Serdar Yüksel

TL;DR

This work analyzes optimal control of continuous-time stochastic diffusions under fully observed, partially observed, and decentralized information structures. It develops a unified probabilistic framework using relaxed wide-sense controls and two methodological routes—measure transformation via Girsanov and a direct approach—to establish existence of optimal policies and near-optimal discrete-time approximations. The core contributions include proving existence of optimal policies in relaxed spaces, demonstrating near-optimality of piecewise-constant controls, and establishing convergence of discrete-time models to their continuous-time counterparts across all information structures. By enabling discrete-time POMDP and decentralized control methods to be applied to continuous-time problems, the results provide practical avenues for computation, learning, and near-optimal design in complex stochastic control settings. The approach also yields a flexible toolkit for extending discrete-time stochastic control theory to richer information patterns and coupled-dynamics scenarios.

Abstract

We present existence and discrete-time approximation results on optimal control policies for continuous-time stochastic control problems under a variety of information structures. These include fully observed models, partially observed models and multi-agent models with decentralized information structures. While there exist comprehensive existence and approximations results for the fully observed setup in the literature, few prior research exists on discrete-time approximation results for partially observed models. For decentralized models, even existence results have not received much attention except for specialized models and approximation has been an open problem. Our existence and approximations results lead to the applicability of well-established partially observed Markov decision processes and the relatively more mature theory of discrete-time decentralized stochastic control to be applicable for computing near optimal solutions for continuous-time stochastic control.

Controlled Diffusions under Full, Partial and Decentralized Information: Existence of Optimal Policies and Discrete-Time Approximations

TL;DR

This work analyzes optimal control of continuous-time stochastic diffusions under fully observed, partially observed, and decentralized information structures. It develops a unified probabilistic framework using relaxed wide-sense controls and two methodological routes—measure transformation via Girsanov and a direct approach—to establish existence of optimal policies and near-optimal discrete-time approximations. The core contributions include proving existence of optimal policies in relaxed spaces, demonstrating near-optimality of piecewise-constant controls, and establishing convergence of discrete-time models to their continuous-time counterparts across all information structures. By enabling discrete-time POMDP and decentralized control methods to be applied to continuous-time problems, the results provide practical avenues for computation, learning, and near-optimal design in complex stochastic control settings. The approach also yields a flexible toolkit for extending discrete-time stochastic control theory to richer information patterns and coupled-dynamics scenarios.

Abstract

We present existence and discrete-time approximation results on optimal control policies for continuous-time stochastic control problems under a variety of information structures. These include fully observed models, partially observed models and multi-agent models with decentralized information structures. While there exist comprehensive existence and approximations results for the fully observed setup in the literature, few prior research exists on discrete-time approximation results for partially observed models. For decentralized models, even existence results have not received much attention except for specialized models and approximation has been an open problem. Our existence and approximations results lead to the applicability of well-established partially observed Markov decision processes and the relatively more mature theory of discrete-time decentralized stochastic control to be applicable for computing near optimal solutions for continuous-time stochastic control.
Paper Structure (17 sections, 15 theorems, 83 equations)

This paper contains 17 sections, 15 theorems, 83 equations.

Table of Contents

  1. Introduction
  2. Fully Observed Model
  3. Two Approaches: With or without Measure Transformation
  4. Without measure transformation
  5. With measure transformation
  6. Partially Observed Setup
  7. Decentralized Information
  8. Decentralized Model with Local Measurements
  9. Decentralized Model with Coupled Dynamics and Local State
  10. Near Optimality of Control Policies Designed for Discrete-time Models via Sampling
  11. Fully Observed Setup
  12. Partially Observed Setup
  13. Decentralized Setup
  14. Decentralized Model with Local Measurements:
  15. Decentralized Model with Coupled Dynamics and Local State
  16. ...and 2 more sections

Key Result

Theorem 1

\newlabelcontConvPathM0 Suppose that Assumptions A1 and A2 hold. If $m^n \to m$ weakly as $n\to \infty$, with $X^n$ is the solution of diffProcess under the policy $m^n$, i.e., it satisfies Then the solution process $X^n$ converges weakly to $X$ which is a unique solution of

Theorems & Definitions (20)

  • Theorem 1
  • Remark 2.1
  • Theorem 2
  • Theorem 3
  • Proof 1
  • Remark 2.2
  • Remark 2.3: Comparison between the approaches: With measure transformation vs. without measure transformation
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • ...and 10 more