Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Design and Evaluation of Whole-Page Experience Optimization for E-commerce Search

Pratik Lahiri, Bingqing Ge, Zhou Qin, Aditya Jumde, Shuning Huo, Lucas Scottini, Yi Liu, Mahmoud Mamlouk, Wenyang Liu

TL;DR

The paper tackles the challenge of optimizing e-commerce SRPs that are now 2D, widget-rich layouts by introducing DV-WPX, a causal framework that maps whole-page quality to downstream value over a $12$-week horizon. It combines a region-aware, quasi-experimental identification strategy with a brand-alignment satisfaction metric (PR-WP-BMR) and a production page-template ranker that jointly optimizes revenue, engagement, and long-term satisfaction via Thompson sampling. The key contributions are the DV-WPX causal framework, the region-based satisfaction metric, and the integration of these signals into a multi-objective template ranker evaluated through offline and online experiments, showing improvements in brand relevance and small but consistent revenue gains, with long-horizon benefits more pronounced when using DV-WPX weights. The work demonstrates a practical, scalable approach to long-term user satisfaction in real-world e-commerce search, with potential extensions to additional quality dimensions and adaptive horizons that can further enhance long-term value.

Abstract

E-commerce Search Results Pages (SRPs) are evolving from linear lists to complex, non-linear layouts, rendering traditional position-biased ranking models insufficient. Moreover, existing optimization frameworks typically maximize short-term signals (e.g., clicks, same-day revenue) because long-term satisfaction metrics (e.g., expected two-week revenue) involve delayed feedback and challenging long-horizon credit attribution. To bridge these gaps, we propose a novel Whole-Page Experience Optimization Framework. Unlike traditional list-wise rankers, our approach explicitly models the interplay between item relevance, 2D positional layout, and visual elements. We use a causal framework to develop metrics for measuring long-term user satisfaction based on quasi-experimental data. We validate our approach through industry-scale A/B testing, where the model demonstrated a 1.86% improvement in brand relevance (our primary customer experience metric) while simultaneously achieving a statistically significant revenue uplift of +0.05%

Design and Evaluation of Whole-Page Experience Optimization for E-commerce Search

TL;DR

The paper tackles the challenge of optimizing e-commerce SRPs that are now 2D, widget-rich layouts by introducing DV-WPX, a causal framework that maps whole-page quality to downstream value over a -week horizon. It combines a region-aware, quasi-experimental identification strategy with a brand-alignment satisfaction metric (PR-WP-BMR) and a production page-template ranker that jointly optimizes revenue, engagement, and long-term satisfaction via Thompson sampling. The key contributions are the DV-WPX causal framework, the region-based satisfaction metric, and the integration of these signals into a multi-objective template ranker evaluated through offline and online experiments, showing improvements in brand relevance and small but consistent revenue gains, with long-horizon benefits more pronounced when using DV-WPX weights. The work demonstrates a practical, scalable approach to long-term user satisfaction in real-world e-commerce search, with potential extensions to additional quality dimensions and adaptive horizons that can further enhance long-term value.

Abstract

E-commerce Search Results Pages (SRPs) are evolving from linear lists to complex, non-linear layouts, rendering traditional position-biased ranking models insufficient. Moreover, existing optimization frameworks typically maximize short-term signals (e.g., clicks, same-day revenue) because long-term satisfaction metrics (e.g., expected two-week revenue) involve delayed feedback and challenging long-horizon credit attribution. To bridge these gaps, we propose a novel Whole-Page Experience Optimization Framework. Unlike traditional list-wise rankers, our approach explicitly models the interplay between item relevance, 2D positional layout, and visual elements. We use a causal framework to develop metrics for measuring long-term user satisfaction based on quasi-experimental data. We validate our approach through industry-scale A/B testing, where the model demonstrated a 1.86% improvement in brand relevance (our primary customer experience metric) while simultaneously achieving a statistically significant revenue uplift of +0.05%
Paper Structure (14 sections, 6 equations, 1 figure, 2 tables)

This paper contains 14 sections, 6 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. DV-WPX: A Causal Inference Framework
  4. User Satisfaction Metric Derived from DV-WPX
  5. Page Template Ranker
  6. Problem Formulation
  7. Input Features
  8. Modeling
  9. Inference and Decision Process
  10. Results
  11. Offline Data and Results
  12. Online Test Results
  13. Discussion
  14. Ethical Considerations

Figures (1)

  • Figure 1: Different search results page layouts. Brand XYZ items are shown in bold purple.