Sim2Act: Robust Simulation-to-Decision Learning via Adversarial Calibration and Group-Relative Perturbation

TL;DR

This work proposes Sim2Act, a robust simulation-to-decision framework that addresses both simulator and policy robustness, and introduces an adversarial calibration mechanism that re-weights simulation errors in decision-critical state-action pairs to align surrogate fidelity with downstream decision impact.

Abstract

Simulation-to-decision learning enables safe policy training in digital environments without risking real-world deployment, and has become essential in mission-critical domains such as supply chains and industrial systems. However, simulators learned from noisy or biased real-world data often exhibit prediction errors in decision-critical regions, leading to unstable action ranking and unreliable policies. Existing approaches either focus on improving average simulation fidelity or adopt conservative regularization, which may cause policy collapse by discarding high-risk high-reward actions. We propose Sim2Act, a robust simulation-to-decision framework that addresses both simulator and policy robustness. First, we introduce an adversarial calibration mechanism that re-weights simulation errors in decision-critical state-action pairs to align surrogate fidelity with downstream decision impact. Second, we develop a group-relative perturbation strategy that stabilizes policy learning under simulator uncertainty without enforcing overly pessimistic constraints. Extensive experiments on multiple supply chain benchmarks demonstrate improved simulation robustness and more stable decision performance under structured and unstructured perturbations.

Figures (5)

• Figure 1: Sim2Act Framework. (a) The overview illustrates the two-stage structure of our framework: simulator calibration and perturbed decision-making. (b) Module details: The simulator, implemented as an LSTM-based encoder-decoder model with linear feature extractor, predicts outcomes from state-action pairs and applies adversarial correction to reduce systematic prediction errors. During decision-making, latent states are perturbed using Gaussian noise derived from the simulator's estimated covariance $\Sigma$, producing a distribution of plausible states. The decision-maker $\mathcal{D}_\theta$ learns a robust policy by maximizing group-relative advantages across these perturbed states, enhancing generalization under latent uncertainty.
• Figure 2: Decision-maker robustness under increasing perturbation across three datasets. Curves show the degradation of overall decision reward as perturbation strength increases, with slopes indicating sensitivity to uncertainty. Sim2Act maintains flatter degradation curves and smaller slopes than baselines, demonstrating stable performance under both latent-structured and unstructured perturbations (Goal 2).
• Figure 3: CVaR@5 Robustness on DataCo, GlobalStore, and OAS datasets. Solid bars denote nominal performance ($p=0$), while hatched bars denote performance under perturbation ($p=0.5$).
• Figure 4: Simulator robustness under perturbation across three datasets. Radar plots summarize worst-case accuracy, variance, and drop rate, Sim2Act improves worst-case accuracy and reduces variability, demonstrating effective decision-critical calibration.
• Figure 5: Ablation study of Sim2Act components across three datasets. The first three panels report decision robustness under perturbation for different module combinations: None (S2D), +SimCal, +DecPert, and +Both. +SimCal improves robustness in decision-critical regions by reducing performance degradation, while +DecPert stabilizes decision behavior under uncertainty. The calibration heatmap (right) visualizes action-level reliability gains, showing that simulator calibration concentrates improvements on decision-critical actions.