Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Malliavin Calculus with Weak Derivatives for Counterfactual Stochastic Optimization

Vikram Krishnamurthy, Luke Snow

TL;DR

This paper tackles counterfactual stochastic optimization for conditional losses under rare conditioning events in diffusion models, where direct sampling is infeasible. It develops a kernel-free two-stage framework: (i) a Malliavin-calculus-based Skorohod representation expresses $\mathbb{E}[\ell(X^{\theta}) \mid g(X^{\theta})=0]$ as a ratio of unconditional expectations, yielding standard $O(1/N)$ MC variance even for measure-zero events, and (ii) a weak-derivative gradient estimator based on the Hahn–Jordan decomposition provides $O(1)$ variance in the time horizon $T$, avoiding the $O(T)$ variance growth of score-function methods. The combination supports an efficient counterfactual stochastic gradient algorithm for approximating local minima of $L(\theta)$, with a concrete implementation for an Ornstein–Uhlenbeck process. The framework connects Malliavin calculus, generator-based sensitivities, and discrete weak-derivative methods to deliver kernel-free, scalable optimization in rare-event regimes, with implications for passive learning and safety-constrained diffusion models.

Abstract

We study counterfactual stochastic optimization of conditional loss functionals under misspecified and noisy gradient information. The difficulty is that when the conditioning event has vanishing or zero probability, naive Monte Carlo estimators are prohibitively inefficient; kernel smoothing, though common, suffers from slow convergence. We propose a two-stage kernel-free methodology. First, we show using Malliavin calculus that the conditional loss functional of a diffusion process admits an exact representation as a Skorohod integral, yielding variance comparable to classical Monte-Carlo variance. Second, we establish that a weak derivative estimate of the conditional loss functional with respect to model parameters can be evaluated with constant variance, in contrast to the widely used score function method whose variance grows linearly in the sample path length. Together, these results yield an efficient framework for counterfactual conditional stochastic gradient algorithms in rare-event regimes.

Malliavin Calculus with Weak Derivatives for Counterfactual Stochastic Optimization

TL;DR

This paper tackles counterfactual stochastic optimization for conditional losses under rare conditioning events in diffusion models, where direct sampling is infeasible. It develops a kernel-free two-stage framework: (i) a Malliavin-calculus-based Skorohod representation expresses as a ratio of unconditional expectations, yielding standard MC variance even for measure-zero events, and (ii) a weak-derivative gradient estimator based on the Hahn–Jordan decomposition provides variance in the time horizon , avoiding the variance growth of score-function methods. The combination supports an efficient counterfactual stochastic gradient algorithm for approximating local minima of , with a concrete implementation for an Ornstein–Uhlenbeck process. The framework connects Malliavin calculus, generator-based sensitivities, and discrete weak-derivative methods to deliver kernel-free, scalable optimization in rare-event regimes, with implications for passive learning and safety-constrained diffusion models.

Abstract

We study counterfactual stochastic optimization of conditional loss functionals under misspecified and noisy gradient information. The difficulty is that when the conditioning event has vanishing or zero probability, naive Monte Carlo estimators are prohibitively inefficient; kernel smoothing, though common, suffers from slow convergence. We propose a two-stage kernel-free methodology. First, we show using Malliavin calculus that the conditional loss functional of a diffusion process admits an exact representation as a Skorohod integral, yielding variance comparable to classical Monte-Carlo variance. Second, we establish that a weak derivative estimate of the conditional loss functional with respect to model parameters can be evaluated with constant variance, in contrast to the widely used score function method whose variance grows linearly in the sample path length. Together, these results yield an efficient framework for counterfactual conditional stochastic gradient algorithms in rare-event regimes.

Paper Structure

This paper contains 19 sections, 2 theorems, 46 equations, 4 figures, 1 algorithm.

Table of Contents

  1. Introduction and Problem Formulation
  2. Main Results
  3. Context. Passive Learning
  4. Malliavin Calculus Approach to Estimate Conditional Loss
  5. Preliminaries. Malliavin Calculus.
  6. Malliavin derivative
  7. Skorohod integral
  8. Integration by parts
  9. Computing Malliavin Derivative and Skorohod Integral
  10. Malliavin Calculus Expression for Conditional Expectation
  11. Weak Derivative Estimator
  12. Discrete-time Weak Derivative of Transition Probabilities
  13. Weak-Derivative Estimator
  14. Connection to Infinitesimal Generator
  15. Infinitesimal Generator Formulation
  16. ...and 4 more sections

Key Result

Theorem 1

Assume $\ell(X^\theta), g(X^\theta) \in L^2(\Omega)$ and $D_t\ell(X^\theta), D_tg(X^\theta) \in L^2(\Omega \times [0,T])$. Then the conditional loss $L$ in eq:equiv_loss is Here $u$ is any process that satisfies

Figures (4)

  • Figure 1: Counterfactual Stochastic Gradient Algorithm
  • Figure 2: Conceptual Schematic of Hahn-Jordan Decomposition for Weak Derivative Estimator. CRN denotes common random number generation.
  • Figure 3: Convergence of Ornstein-Uhlenbeck Malliavin quotient \ref{['eq:mall_ce']}, with respect to simulated paths $N$. We see that \ref{['eq:mall_ce']} recovers a $O(N^{-1/2})$ convergence rate even though we condition on a measure-zero event.
  • Figure 4: Variance scaling of the weak derivative estimator and the score function estimator, for varying time horizon $T$. We verify the stable $O(1)$ variance scaling of the weak derivative estimator, in contrast to the $O(T)$ variance scaling of the score function estimator.

Theorems & Definitions (3)

  • Theorem 1
  • Theorem 2: Discrete-Time Hahn--Jordan Weak Derivative
  • proof