Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic Trajectory Influence Functions for LQR: Joint Sensitivity Through Dynamics and Noise Covariance

Jiachen Li, Shihao Li, Soovadeep Bakshi, Jiamin Xu, Dongmei Chen

Abstract

Model-based controllers learned from data have the biases and noise of their training trajectories, making it important to know which trajectories help or hurt closed-loop performance. Influence functions, widely used in machine learning for data attribution, approximate this effect through first-order parameter-shift surrogates, avoiding costly retraining. Applying them to stochastic LQR, however, is nontrivial because the cost depends on the learned dynamics through the Riccati equation, and the process-noise covariance is estimated from the same residuals. We develop a three-level influence hierarchy that accounts for both channels.

Stochastic Trajectory Influence Functions for LQR: Joint Sensitivity Through Dynamics and Noise Covariance

Abstract

Model-based controllers learned from data have the biases and noise of their training trajectories, making it important to know which trajectories help or hurt closed-loop performance. Influence functions, widely used in machine learning for data attribution, approximate this effect through first-order parameter-shift surrogates, avoiding costly retraining. Applying them to stochastic LQR, however, is nontrivial because the cost depends on the learned dynamics through the Riccati equation, and the process-noise covariance is estimated from the same residuals. We develop a three-level influence hierarchy that accounts for both channels.
Paper Structure (11 sections, 11 theorems, 51 equations, 2 figures, 1 table, 1 algorithm)

This paper contains 11 sections, 11 theorems, 51 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. System Identification Model
  4. Riccati Equation and Notation
  5. Plug-in Stochastic Cost Functional
  6. Leave-One-Trajectory-Out Covariance Analysis
  7. Stochastic Control Influence Score
  8. Numerical Experiments
  9. Experimental Setup
  10. Results and Discussion
  11. Conclusion

Key Result

Proposition 1

Under Definition def:loto_est, where $F_{\setminus k}$ denotes the reduced objective. Consequently, the first-order Newton step at $\hat{\theta}$ is

Figures (2)

  • Figure 1: Method overview.
  • Figure 2: Predicted $\mathrm{IF}^{\mathrm{stoch}}_k$ versus exact LOTO cost change $\Delta\hat{J}_k$ for all four systems.

Theorems & Definitions (27)

  • Remark 1: Stationarity condition
  • Definition 1: Renormalized LOTO estimator
  • Proposition 1: Renormalized LOTO gradient
  • proof
  • Definition 2: Model-side influence (IF$^{\mathrm m}$)
  • Remark 2: Simpler surrogate $H^{-1}g_k$
  • Lemma 1: Riccati gradient IF2_reference
  • Definition 3: Fixed-covariance control influence
  • Remark 3: Plug-in target
  • Proposition 2: Stationary average-cost identity
  • ...and 17 more