Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Ego-Cluster for Network Effect Measurement

Wentao Su, Weitao Duan

TL;DR

The paper tackles the challenge of measuring creator-side effects under network interference in large social networks. It introduces an enhanced Ego Cluster tool based on one-degree label propagation to form ego-centric clusters, enabling scalable cluster-level experiments that isolate creator-level impacts. Key contributions include a ~5× increase in cluster samples, faster runtimes via Spark, and a bias-correction framework for leftover traffic that improves measurement accuracy. A live deployment applied to a new feed-ranking model yielded a 5% lift in a creator-centric surrogate and a modest rise in creator activity, validating the approach's practical utility. Overall, the work provides a robust, scalable methodology for estimating network effects on creator metrics, with clear implications for accelerating creator-focused experimentation in large platforms.

Abstract

The network effect, wherein one user's activity impacts another user, is common in social network platforms. Many new features in social networks are specifically designed to create a network effect, enhancing user engagement. For instance, content creators tend to produce more when their articles and posts receive positive feedback from followers. This paper discusses a new cluster-level experimentation methodology for measuring creator-side metrics in the context of A/B experiments. The methodology is designed to address cases where the experiment randomization unit and the metric measurement unit differ. It is a crucial part of LinkedIn's overall strategy to foster a robust creator community and ecosystem. The method is developed based on widely-cited research at LinkedIn but significantly improves the efficiency and flexibility of the clustering algorithm. This improvement results in a stronger capability for measuring creator-side metrics and an increased velocity for creator-related experiments.

Improving Ego-Cluster for Network Effect Measurement

TL;DR

The paper tackles the challenge of measuring creator-side effects under network interference in large social networks. It introduces an enhanced Ego Cluster tool based on one-degree label propagation to form ego-centric clusters, enabling scalable cluster-level experiments that isolate creator-level impacts. Key contributions include a ~5× increase in cluster samples, faster runtimes via Spark, and a bias-correction framework for leftover traffic that improves measurement accuracy. A live deployment applied to a new feed-ranking model yielded a 5% lift in a creator-centric surrogate and a modest rise in creator activity, validating the approach's practical utility. Overall, the work provides a robust, scalable methodology for estimating network effects on creator metrics, with clear implications for accelerating creator-focused experimentation in large platforms.

Abstract

The network effect, wherein one user's activity impacts another user, is common in social network platforms. Many new features in social networks are specifically designed to create a network effect, enhancing user engagement. For instance, content creators tend to produce more when their articles and posts receive positive feedback from followers. This paper discusses a new cluster-level experimentation methodology for measuring creator-side metrics in the context of A/B experiments. The methodology is designed to address cases where the experiment randomization unit and the metric measurement unit differ. It is a crucial part of LinkedIn's overall strategy to foster a robust creator community and ecosystem. The method is developed based on widely-cited research at LinkedIn but significantly improves the efficiency and flexibility of the clustering algorithm. This improvement results in a stronger capability for measuring creator-side metrics and an increased velocity for creator-related experiments.
Paper Structure (16 sections, 1 theorem, 13 equations, 8 figures, 4 tables)

This paper contains 16 sections, 1 theorem, 13 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Review on Controlled Experiments
  3. Existing Solutions
  4. A New Ego Cluster Tool
  5. Ego Cluster Experimentation Process
  6. New Tool Architecture
  7. Clustering Algorithm
  8. Diagnostic Analysis
  9. cluster quality metrics
  10. network type summary table
  11. Viewer-side Experiment Measurement Correction
  12. Ego Cluster leftover traffic quality
  13. model-free bias correction method
  14. Application and Results
  15. Conclusion
  16. ...and 1 more sections

Key Result

Lemma A.1

Given a directed social network graph G, the algorithm discussed in section sec:algorithm will minimize the spillover effect for cluster creations in an A/B experiment.

Figures (8)

  • Figure 1: cluster-level experimentation process
  • Figure 2: new tool architecture
  • Figure 3: alter assignment based on network aggregation count
  • Figure 4: an example for loss rate calculation
  • Figure 5: connection # difference between Ego Cluster and its leftover traffic
  • ...and 3 more figures

Theorems & Definitions (1)

  • Lemma A.1