Beyond Binary Opinions: A Deep Reinforcement Learning-Based Approach to Uncertainty-Aware Competitive Influence Maximization

TL;DR

This work addresses CIM under realistic opinion uncertainty by moving beyond binary beliefs and modeling uncertain opinions with Subjective Logic. It introduces DRIM, a dual-agent DRL framework that optimizes seed sets while incorporating uncertainty into opinion dynamics and exploration–exploitation decisions, including uncertainty-aware seed selection and partially observable networks. Key contributions include the Uncertainty-based Opinion Model (UOM), integration of Evidential Deep Learning for uncertainty estimation, and multiple uncertainty-aware EE strategies, all validated on three real OSN datasets with strong performance gains and efficiency improvements. The approach advances practical misinformation mitigation and seed-optimization in complex, dynamic networks, offering robust, scalable strategies for real-world CIM scenarios.

Abstract

The Competitive Influence Maximization (CIM) problem involves multiple entities competing for influence in online social networks (OSNs). While Deep Reinforcement Learning (DRL) has shown promise, existing methods often assume users' opinions are binary and ignore their behavior and prior knowledge. We propose DRIM, a multi-dimensional uncertainty-aware DRL-based CIM framework that leverages Subjective Logic (SL) to model uncertainty in user opinions, preferences, and DRL decision-making. DRIM introduces an Uncertainty-based Opinion Model (UOM) for a more realistic representation of user uncertainty and optimizes seed selection for propagating true information while countering false information. In addition, it quantifies uncertainty in balancing exploration and exploitation. Results show that UOM significantly enhances true information spread and maintains influence against advanced false information strategies. DRIM-based CIM schemes outperform state-of-the-art methods by up to 57% and 88% in influence while being up to 48% and 77% faster. Sensitivity analysis indicates that higher network observability and greater information propagation boost performance, while high network activity mitigates the effect of users' initial biases.

Figures (12)

• Figure 1: Overview of the CIM problem with Subjective Logic. An SL-based competitive network visualizes belief strength via a color gradient where lighter shades indicate higher uncertainty, with white for neutrals. Blue represents true information supporters (darkest: TIP), and red denotes false information supporters (darkest: FIP). Each user $i$'s opinion is represented by $\omega_i = \{b_i, d_i, u_i\}$, we ignore prior belief $a_i$ because it is predefined as a constant for all users.
• Figure 2: Overview of the Proposed Uncertainty-Aware DRL-based Node Selection: Each agent (i.e., DRL agents for TP and FP, respectively) processes OSN states through a policy network to determine the action distribution. Vacuity and/or dissonance guide the exploration-exploitation decision-making: low uncertainty favors exploitation (selecting the highest-probability action for maximum influence), while high uncertainty promotes exploration (random action selection). The policy network is iteratively updated based on rewards.
• Figure 3: Performance comparison of the considered CIM algorithms with respect to % of nodes in the true party (TP), representing TP's influence.
• Figure 4: TP's influence across CIM algorithms when TP and FP use DRL for seed selection in the URV Email network. #IP denotes the number of information propagation by TP, 'NO' represents network observability %, and PB indicates users' prior belief in true information (i.e., $a$). In all cases except (c), we set $a=0.5$.
• Figure 5: TP's influence under various EE strategies under the three networks.