Approximate Dynamic Programming with Feasibility Guarantees

TL;DR

The paper develops a feasibility-preserving approximate dynamic programming framework for tree-structured optimization in networked systems, guaranteeing a feasible solution in one backward-forward pass even with nonconvex constraints. It unifies DP concepts with projection-based feasibility, introducing inner approximations to handle intractable constraint sets while preserving feasibility through all subproblems. The FP-ADP scheme enables practical computation by detailing exact/projection-based set representations, sampling-based approximations, and design-centered ellipsoidal inner approximations, demonstrated on optimal control and power-systems OPF problems. The results highlight a principled balance between feasibility guarantees and suboptimality, with strong potential for cross-domain method transfer in hierarchical optimization. The work provides a concrete pathway to scalable, safe operation of complex infrastructure through structured decomposition and projection-based feasibility.

Abstract

Safe and economic operation of networked systems is often challenging. Optimization-based schemes are frequently considered, since they achieve near-optimality while ensuring safety via the explicit consideration of constraints. In applications, these schemes, however, often require solving large-scale optimization problems. Iterative techniques from distributed optimization are frequently proposed for complexity reduction. Yet, they achieve feasibility only asymptotically, which induces a substantial computational burden. This work presents an approximate dynamic programming scheme, which is guaranteed to deliver a feasible solution in "one shot", i.e., in one backward-forward iteration over all subproblems provided they are coupled by a tree structure. Our proposed scheme generalizes methods from seemingly disconnected domains such as power systems and optimal control. We demonstrate its efficacy for problems with nonconvex constraints via numerical examples from both domains.

Figures (7)

• Figure 1: Example for a general (left) and a tree-structured (right) interaction graph.
• Figure 2: Classic serial DP example.
• Figure 3: Feasibility-preserving ADP example.
• Figure 4: FP-ADP for OCP \ref{['eq:OCP']}, where $\mathcal{S}$ are time instances and $G$ is a path graph.
• Figure 5: Modified distribution grid model from Grady1992.

Theorems & Definitions (13)

• Example 1: Interaction graph
• Remark 1: Alternative graph encoding techniques
• Lemma 1: Value function formulation of problem \ref{['eq:sepNLP1']}
• Remark 2: Serial and nonserial dynamic programming
• Lemma 2: Computing domains via projection
• proof
• Theorem 1: Constraint consideration via projections
• proof
• Theorem 2: Feasibility in \ref{['alg:appDP']}
• proof