Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Valuing an Engagement Surface using a Large Scale Dynamic Causal Model

Abhimanyu Mukerji, Sushant More, Ashwin Viswanathan Kannan, Lakshmi Ravi, Hua Chen, Naman Kohli, Chris Khawand, Dinesh Mandalapu

TL;DR

This work addresses the challenge of valuing Engagement Surfaces in online shopping by developing a large-scale Dynamic Causal Model that estimates long-term, mediated effects from observational data. The approach models time-evolving interactions among thousands of customer surrogates and outcomes, extends to same-period (near-simultaneous) effects, and produces counterfactual valuations of ES impact via sequential shocks. Valuation is operationalized through Shapley attribution and a Spark-based, modular system that supports scalable training, scoring, and interpretability, yielding insights on channels, product groups, and key ES features. Practically, the framework enables ROI assessment and strategic prioritization of ES features, while highlighting the importance of contemporaneous effects in accurately quantifying ES value.

Abstract

With recent rapid growth in online shopping, AI-powered Engagement Surfaces (ES) have become ubiquitous across retail services. These engagement surfaces perform an increasing range of functions, including recommending new products for purchase, reminding customers of their orders and providing delivery notifications. Understanding the causal effect of engagement surfaces on value driven for customers and businesses remains an open scientific question. In this paper, we develop a dynamic causal model at scale to disentangle value attributable to an ES, and to assess its effectiveness. We demonstrate the application of this model to inform business decision-making by understanding returns on investment in the ES, and identifying product lines and features where the ES adds the most value.

Valuing an Engagement Surface using a Large Scale Dynamic Causal Model

TL;DR

This work addresses the challenge of valuing Engagement Surfaces in online shopping by developing a large-scale Dynamic Causal Model that estimates long-term, mediated effects from observational data. The approach models time-evolving interactions among thousands of customer surrogates and outcomes, extends to same-period (near-simultaneous) effects, and produces counterfactual valuations of ES impact via sequential shocks. Valuation is operationalized through Shapley attribution and a Spark-based, modular system that supports scalable training, scoring, and interpretability, yielding insights on channels, product groups, and key ES features. Practically, the framework enables ROI assessment and strategic prioritization of ES features, while highlighting the importance of contemporaneous effects in accurately quantifying ES value.

Abstract

With recent rapid growth in online shopping, AI-powered Engagement Surfaces (ES) have become ubiquitous across retail services. These engagement surfaces perform an increasing range of functions, including recommending new products for purchase, reminding customers of their orders and providing delivery notifications. Understanding the causal effect of engagement surfaces on value driven for customers and businesses remains an open scientific question. In this paper, we develop a dynamic causal model at scale to disentangle value attributable to an ES, and to assess its effectiveness. We demonstrate the application of this model to inform business decision-making by understanding returns on investment in the ES, and identifying product lines and features where the ES adds the most value.
Paper Structure (21 sections, 7 equations, 5 figures, 5 tables)

This paper contains 21 sections, 7 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Dynamic Causal Model
  3. DCM Training
  4. DCM Scoring
  5. Contemporaneous treatment effects
  6. Business Motivation
  7. Mathematical Framework
  8. Implementation overview
  9. Shapley Attribution
  10. System Architecture Overview
  11. Results
  12. 1. Top channel drives over 50% of ES value:
  13. 2. The ES drives the highest value in low average selling price (ASP) and regular purchase categories:
  14. 3. Top two features drive 73% of ES's impact:
  15. 4. Near-simultaneity of ES interaction and purchases plays a large role in value driven:
  16. ...and 6 more sections

Figures (5)

  • Figure 1: DCM Causal Graph: 3-period example. The solid arrows represent direct relationships. The dashed arrow represent indirect (mediated) relationships.
  • Figure 2: Simplified representation of the DCM causal graph with and without the same-period effects.
  • Figure 3: DCM framework architecture with training and scoring stages.
  • Figure 4: JSON config snippet depicting the regression structure in the standard DCM framework.
  • Figure 5: JSON config snippet depicting the regression structure in the DCM framework with same-period effects included.