Consistent inverse optimal control for infinite time-horizon discounted nonlinear systems under noisy observations

TL;DR

This work tackles inverse optimal control for discrete‑time, infinite‑horizon discounted nonlinear systems observed with noise. It builds a convex IOC framework based on occupation measures to handle weakly Feller dynamics, recasting IOC as an infinite‑dimensional linear program and approximating it with polynomial bases. A two‑step misspecified GMM estimator robustly recovers the occupation‑measure moments from noisy data, enabling a finite‑dimensional, convex optimization to infer the cost parameters $\theta_\ell$ from $\ell = \theta_\ell^\top \varphi$. The authors prove statistical and asymptotic consistency and demonstrate accurate cost recovery in both linear and nonlinear experiments under noise, highlighting practical viability for robust IOC in real‑world sensing scenarios.

Abstract

Inverse optimal control (IOC) aims to estimate the underlying cost that governs the observed behavior of an expert system. However, in practical scenarios, the collected data is often corrupted by noise, which poses significant challenges for accurate cost function recovery. In this work, we propose an IOC framework that effectively addresses the presence of observation noise. In particular, compared to our previous work \cite{wang2025consistent}, we consider the case of discrete-time, infinite-horizon, discounted MDPs whose transition kernel is only weak Feller. By leveraging the occupation measure framework, we first establish the necessary and sufficient optimality conditions for the expert policy and then construct an infinite dimensional optimization problem based on these conditions. This problem is then approximated by polynomials to get a finite-dimensional numerically solvable one, which relies on the moments of the state-action trajectory's occupation measure. More specifically, the moments are robustly estimated from the noisy observations by a combined misspecified Generalized Method of Moments (GMM) estimator derived from observation model and system dynamics. Consequently, the entire algorithm is based on convex optimization which alleviates the issues that arise from local minima and is asymptotically and statistically consistent. Finally, the performance of the proposed method is illustrated through numerical examples.

Figures (3)

• Figure 1: Signed estimation error distributions for $(q_1, q_2)$ over 100 trials. Histogram bins span the 1st--99th percentiles (approximately 2% outliers excluded). Both distributions center at zero with small variance, validating the proposed IOC method.
• Figure 2: Estimation error vs. dataset size under three observation noise levels. Lines represent mean errors across 400 trials; higher noise levels produce consistently larger errors. As the dataset increases, the estimation error first decreases and then tends to stabilize.
• Figure 3: Estimation error vs. dataset size for three polynomial basis configurations. Higher-degree bases yield better accuracy with large datasets, while for small datasets where statistical noise dominates, the performance is similar.

Theorems & Definitions (12)

• Proposition 3.1
• proof
• Remark 3.2: Extension of our prior work
• Proposition 3.3
• Lemma 4.1
• proof
• Lemma 4.2
• proof
• Lemma 4.3
• proof