A Framework for Personalized Persuasiveness Prediction via Context-Aware User Profiling

TL;DR

The paper tackles personalized persuasiveness prediction by introducing a context-aware user profiling framework that learns to retrieve and summarize persuasion-relevant records from a user’s history. It couples a trainable query generator with a trainable profiler, both optimized via Direct Preference Optimization, to produce a context-conditioned user profile that informs a predictor model. Empirical results on the ChangeMyView dataset show consistent improvements over baselines in end-to-end view-change prediction, with notable gains when employing task-oriented profiles and persuasion-aware retrieval. The findings underscore that effective persuasion modeling depends on context-dependent, predictor-specific user representations rather than static demographic attributes, offering a scalable approach for personalized decision-support systems.

Abstract

Estimating the persuasiveness of messages is critical in various applications, from recommender systems to safety assessment of LLMs. While it is imperative to consider the target persuadee's characteristics, such as their values, experiences, and reasoning styles, there is currently no established systematic framework to optimize leveraging a persuadee's past activities (e.g., conversations) to the benefit of a persuasiveness prediction model. To address this problem, we propose a context-aware user profiling framework with two trainable components: a query generator that generates optimal queries to retrieve persuasion-relevant records from a user's history, and a profiler that summarizes these records into a profile to effectively inform the persuasiveness prediction model. Our evaluation on the ChangeMyView Reddit dataset shows consistent improvements over existing methods across multiple predictor models, with gains of up to +13.77%p in F1 score. Further analysis shows that effective user profiles are context-dependent and predictor-specific, rather than relying on static attributes or surface-level similarity. Together, these results highlight the importance of task-oriented, context-dependent user profiling for personalized persuasiveness prediction.

Figures (10)

• Figure 1: Overview of the view-change prediction task with context-aware user profiling on CMV. Given an original post, the system (a) retrieves relevant user records, (b) constructs a textual user profile, and (c) predicts whether a comment will change the user’s view.
• Figure 2: Overview of the proposed framework. Flame and snowflake icons denote trainable and frozen models, respectively. QG denotes our query generator model. The top illustrates the inference pipeline with three stages: (a) retrieval, (b) profiling, and (c) view-change prediction. The bottom shows the training pipeline, consisting of (d) profiler training, (e) record-level persuasion utility scoring, and (f) query generator training.
• Figure 3: F1 by topic and claim type of the post, comparing the original and trained profilers. Llama-3.1-8B-Instruct is used as the predictor model.
• Figure 4: Analysis of profile-dimension frequency shifts ($\Delta$DF) and performance gains ($\Delta$F1) between the original and trained profilers. (a) Correlation between $\Delta$DF and $\Delta$F1. (b) $\Delta$DF for cases with $\Delta$F1 $>$ 0. Llama-3.1-8B-Instruct is used as the predictor model.
• Figure 5: Examples of user profiles generated by our trained profiler when used Llama3.1 8B Instruct as the predictor model for training.