Decision Oriented Technique (DOTechnique): Finding Model Validity Through Decision-Maker Context

TL;DR

DOTechnique reframes model validity around decision consistency rather than output similarity, enabling identification of validity regions even when explicit validity frames are unavailable. By defining a decision space $\mathcal{Y}$ and a decision-distance $d_Y$, and by using a binary-search-enabled boundary search guided by domain constraints and symbolic reasoning, the method efficiently narrows validity regions. The highway lane-change case demonstrates how surrogate models can be judged equivalent in decision outcomes to a high-fidelity model within a computed region $\mathcal{V_\varepsilon}$, illustrating practical gains in computational efficiency. The work emphasizes decision-maker context as a viable basis for validity and lays groundwork for broader validation and extension across domains.

Abstract

Model validity is as critical as the model itself, especially when guiding decision-making processes. Traditional approaches often rely on predefined validity frames, which may not always be available or sufficient. This paper introduces the Decision Oriented Technique (DOTechnique), a novel method for determining model validity based on decision consistency rather than output similarity. By evaluating whether surrogate models lead to equivalent decisions compared to high-fidelity models, DOTechnique enables efficient identification of validity regions, even in the absence of explicit validity boundaries. The approach integrates domain constraints and symbolic reasoning to narrow the search space, enhancing computational efficiency. A highway lane change system serves as a motivating example, demonstrating how DOTechnique can uncover the validity region of a simulation model. The results highlight the potential of the technique to support finding model validity through decision-maker context.

Figures (5)

• Figure 1: Conceptual framework.
• Figure 2: Lane changing scenario.
• Figure 3: Validity region of the high-validity model.
• Figure 4: Finding model validity scenario.
• Figure 5: Constant Acceleration (C.A) model validity.