Data-Driven Conditional Flexibility Index

TL;DR

The paper tackles robust scheduling under uncertainty by extending the classical flexibility index to a conditional, data-driven framework called the conditional flexibility index (CFI). It uses a conditional normalizing flow to learn a latent-space hypersphere that is mapped to the data space, conditioning the admissible uncertainty set on contextual information and enabling a probabilistic interpretation via mass preservation in the latent space. The approach is embedded in a generalized semi-infinite programming formulation and solved with adaptive discretization, demonstrated on illustrative datasets and a security-constrained unit commitment (SCUC) problem, showing that context-aware sets can improve feasibility and scheduling quality while highlighting variability and computational considerations. The results indicate that while data-driven and conditional sets do not universally outperform simple sets, they offer principled means to exclude regions with no historical realizations and to adapt uncertainty sets to real-world context, with notable gains when temporal context is informative. Overall, CFI provides a flexible, context-aware tool for conditional robust optimization in power systems and beyond, at the cost of increased model complexity and solver effort; future work could further improve scalability and conditional coverage through algorithmic advances and richer contextual features.

Abstract

With the increasing flexibilization of processes, determining robust scheduling decisions has become an important goal. Traditionally, the flexibility index has been used to identify safe operating schedules by approximating the admissible uncertainty region using simple admissible uncertainty sets, such as hypercubes. Presently, available contextual information, such as forecasts, has not been considered to define the admissible uncertainty set when determining the flexibility index. We propose the conditional flexibility index (CFI), which extends the traditional flexibility index in two ways: by learning the parametrized admissible uncertainty set from historical data and by using contextual information to make the admissible uncertainty set conditional. This is achieved using a normalizing flow that learns a bijective mapping from a Gaussian base distribution to the data distribution. The admissible latent uncertainty set is constructed as a hypersphere in the latent space and mapped to the data space. By incorporating contextual information, the CFI provides a more informative estimate of flexibility by defining admissible uncertainty sets in regions that are more likely to be relevant under given conditions. Using an illustrative example, we show that no general statement can be made about data-driven admissible uncertainty sets outperforming simple sets, or conditional sets outperforming unconditional ones. However, both data-driven and conditional admissible uncertainty sets ensure that only regions of the uncertain parameter space containing realizations are considered. We apply the CFI to a security-constrained unit commitment example and demonstrate that the CFI can improve scheduling quality by incorporating temporal information.

Figures (16)

• Figure 1: Illustration of the flexibility index problem. The white area represents the admissible uncertainty region, while shaded areas indicate infeasible parameter realizations. Nominal parameter realizations $y_{i,n}$ are marked. The true uncertainty set comprises all possible uncertain parameter realizations and is unknown in practice; it is shown here only for visualization. The left panel illustrates the traditional approach of approximating the admissible uncertainty region with a hypercube without any knowledge of the underlying true uncertainty set. The admissible uncertainty set contains regions where no uncertainty realizations occur. The middle panel shows improved approximation when correct parameter correlations are assumed; in this special case, the admissible uncertainty set coincides with the true uncertainty set, which is not generally the case. The right panel shows uncertain parameter realizations (black dots) used to estimate parameter correlations.
• Figure 2: Illustration of a single RealNVP transformation block. The input $\mathbf{l}_{i}$ is split into two parts (masking): $\mathbf{l}_{i, 1}$ and $\mathbf{l}_{i, 2}$. The first part $\mathbf{l}_{i, 1}$ remains unchanged and is used to parameterize an affine transformation of $\mathbf{l}_{i, 2}$. The scale and translation parameters $\mathbf{s}$ and $\mathbf{t}$ are determined using an artificial neural network (ANN) conditioned on contextual information $\mathbf{c}$. The ANN comprises a multi-layer perception (MLP) whose outputs are split into the translation parameter $\mathbf{t}$ and a second component used to compute the scale $\mathbf{s}$. The scale vector is obtained by applying a softplus activation, adding a bias term $\mathbf{b}$, and clipping the result to the range $[0, 3]$. The transfomed output $\mathbf{l}_{i + 1, 2}$ is then combined with the untransformed part of the input $\mathbf{l}_{i + 1, 1}$ to form the output of the RealNVP block $\mathbf{l}_{i + 1}$.
• Figure 3: "Two-Moons" scikit-learn dataset for uncertain parameters, with a noise-level of $0.1$, scaled by a factor of $4$ and shifted by $-2.7,-0.85^T$. The blue crescent moon is associated with the contextual information $c=0$ and the orange moon with $c=1$. $y_1$ and $y_2$ are the two dimensions of the uncertain variable vector $\mathbf{y}$.
• Figure 4: Illustration of the transformation of the admissible uncertainty set from the latent space to the data space. Color indicates the radius of the points in the latent space.
• Figure 5: Unconditional normalizing flow-based and hypercube admissible uncertainty sets plotted in the data space. Colored points indicate samples from the data distribution. Blue color corresponds to $c=0$ and orange color corresponds to $c=1$. Red areas mark infeasible regions.