Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Peer Influence Probabilities with Linear Contextual Bandits

Ahmed Sayeed Faruk, Mohammad Shahverdikondori, Elena Zheleva

TL;DR

This work addresses online learning of heterogeneous peer influence probabilities in networks by framing the problem as a bi-objective contextual linear bandit with two competing goals: minimize cumulative regret $\mathrm{Regret}_T$ and minimize estimation error $\mathrm{RMSE}_T$. It establishes a fundamental trade-off and Pareto frontier between these objectives, proving that no algorithm can optimally achieve both simultaneously. The authors introduce Influence Contextual Bandit ($InfluenceCB$), which alternates between uncertainty-guided exploration and reward-guided exploitation, with a tunable parameter $\beta$ to navigate the frontier and provide theoretical guarantees in the linear setting: $\mathrm{Regret}_T = \tilde{\mathcal{O}}(T^{2\beta})$ and $\mathrm{RMSE}_T = \tilde{\mathcal{O}}(T^{-\beta})$. Empirically, on semi-synthetic networks built from real data, $InfluenceCB$ outperforms observational baselines and standard bandits by offering a controllable trade-off between learning accuracy and cost, with practical implications for diffusion modeling and targeted interventions.

Abstract

In networked environments, users frequently share recommendations about content, products, services, and courses of action with others. The extent to which such recommendations are successful and adopted is highly contextual, dependent on the characteristics of the sender, recipient, their relationship, the recommended item, and the medium, which makes peer influence probabilities highly heterogeneous. Accurate estimation of these probabilities is key to understanding information diffusion processes and to improving the effectiveness of viral marketing strategies. However, learning these probabilities from data is challenging; static data may capture correlations between peer recommendations and peer actions but fails to reveal influence relationships. Online learning algorithms can learn these probabilities from interventions but either waste resources by learning from random exploration or optimize for rewards, thus favoring exploration of the space with higher influence probabilities. In this work, we study learning peer influence probabilities under a contextual linear bandit framework. We show that a fundamental trade-off can arise between regret minimization and estimation error, characterize all achievable rate pairs, and propose an uncertainty-guided exploration algorithm that, by tuning a parameter, attains any pair within this trade-off. Our experiments on semi-synthetic network datasets show the advantages of our method over static methods and contextual bandits that ignore this trade-off.

Learning Peer Influence Probabilities with Linear Contextual Bandits

TL;DR

This work addresses online learning of heterogeneous peer influence probabilities in networks by framing the problem as a bi-objective contextual linear bandit with two competing goals: minimize cumulative regret and minimize estimation error . It establishes a fundamental trade-off and Pareto frontier between these objectives, proving that no algorithm can optimally achieve both simultaneously. The authors introduce Influence Contextual Bandit (), which alternates between uncertainty-guided exploration and reward-guided exploitation, with a tunable parameter to navigate the frontier and provide theoretical guarantees in the linear setting: and . Empirically, on semi-synthetic networks built from real data, outperforms observational baselines and standard bandits by offering a controllable trade-off between learning accuracy and cost, with practical implications for diffusion modeling and targeted interventions.

Abstract

In networked environments, users frequently share recommendations about content, products, services, and courses of action with others. The extent to which such recommendations are successful and adopted is highly contextual, dependent on the characteristics of the sender, recipient, their relationship, the recommended item, and the medium, which makes peer influence probabilities highly heterogeneous. Accurate estimation of these probabilities is key to understanding information diffusion processes and to improving the effectiveness of viral marketing strategies. However, learning these probabilities from data is challenging; static data may capture correlations between peer recommendations and peer actions but fails to reveal influence relationships. Online learning algorithms can learn these probabilities from interventions but either waste resources by learning from random exploration or optimize for rewards, thus favoring exploration of the space with higher influence probabilities. In this work, we study learning peer influence probabilities under a contextual linear bandit framework. We show that a fundamental trade-off can arise between regret minimization and estimation error, characterize all achievable rate pairs, and propose an uncertainty-guided exploration algorithm that, by tuning a parameter, attains any pair within this trade-off. Our experiments on semi-synthetic network datasets show the advantages of our method over static methods and contextual bandits that ignore this trade-off.
Paper Structure (32 sections, 3 theorems, 15 equations, 4 figures, 1 table, 2 algorithms)

This paper contains 32 sections, 3 theorems, 15 equations, 4 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Description
  4. Estimation-Regret Trade-Off
  5. Influence Contextual Bandit framework
  6. Uncertainty-guided Exploration
  7. Reward-guided Exploitation
  8. Illustration of the $InfluenceCB$ Algorithm
  9. Experiments
  10. Data representation
  11. Construction of edge feature representations
  12. Generation of synthetic ground truth peer influence probabilities (linear).
  13. Evaluation metrics
  14. $Regret_T$.
  15. $RMSE_T$
  16. ...and 17 more sections

Key Result

Lemma 4.1

For any instance $(\mathcal{A}, \theta^*) \in \mathcal{E}$, consider the algorithm that allocates its $kT$ selections proportionally to $\mathbf{w}_k^*$, the optimizer of eq: gk-optimal. Then the final (expected) RMSE satisfies where $\tilde{\mathcal{O}}(\cdot)$ hides logarithmic factors.

Figures (4)

  • Figure 1: Trade-off between cumulative regret ($Regret_T$) and estimation error ($RMSE_T$) in $InfluenceCB(\beta, C)$ across linear network-based settings on the semi-synthetic datasets. The results illustrate how increasing $C$ shifts the trade-off: larger $C$ consistently reduces $Regret_T$ but leads to higher $RMSE_T$, with the trade-off depending on the choice of $\beta$.
  • Figure 2: Tradeoff between $Regret_T$ (solid) and $RMSE_T$ (dotted) for $InfluenceCB$ variants compared to $LinUCB$ for varying $k$.
  • Figure 3: Pareto frontiers of cumulative regret $Regret_T$ versus estimation error $RMSE_T$ for $\mathsf{InfluenceCB}(\beta, \mathcal{O})$ compared to baseline bandit and online models on semi-synthetic datasets. $\mathsf{InfluenceCB}$ expands the Pareto frontier and enables tunable trade-offs, with a notably smoother frontier on BlogCatalog. Varying $\beta$ and $C$ shifts the curve between low-regret and low-error regimes.
  • Figure 4: Trade-off between cumulative regret ($Regret_T$) and estimation error ($RMSE_T$) in $InfluenceCB(\beta, C)$ under a non-linear network-based setting. Peer influence probabilities are generated via a feed-forward neural network to simulate model misspecification. Results on Flickr and BlogCatalog show that while $Regret_T$ trends remain consistent with the linear case, $RMSE_T$ exhibits more irregular patterns as $\beta$ and $C$ vary.

Theorems & Definitions (3)

  • Lemma 4.1: RMSE Upper Bound
  • Theorem 4.2: RMSE--Regret Trade-Off
  • Theorem 5.1: Achievability of the Pareto-Frontier