PHORECAST: Enabling AI Understanding of Public Health Outreach Across Populations

TL;DR

PHORECAST presents a large multimodal dataset that ties public health campaign media to rich participant profiles (demographics, personality, locus of control) to enable fine-grained prediction of both individual and community responses to health messaging. By applying LoRA-based fine-tuning and feature-randomization, the study benchmarks vision-language models on predicting opinion indicators and free-form responses, revealing substantial improvements over baselines and highlighting the value of demographic and psychographic conditioning. The work includes thorough ablations showing that education, personality facets, LOC, and in-context Q/A cues differentially drive predictive accuracy, and demonstrates the important role of visual stimuli in shaping precise judgments. While demonstrating promising gains and a path toward socially aware, personalized public health AI, the paper also discusses limitations (e.g., English-speaking, United States–centric samples) and points to future work on temporal dynamics and cross-cultural generalization to broaden applicability.

Abstract

Understanding how diverse individuals and communities respond to persuasive messaging holds significant potential for advancing personalized and socially aware machine learning. While Large Vision and Language Models (VLMs) offer promise, their ability to emulate nuanced, heterogeneous human responses, particularly in high stakes domains like public health, remains underexplored due in part to the lack of comprehensive, multimodal dataset. We introduce PHORECAST (Public Health Outreach REceptivity and CAmpaign Signal Tracking), a multimodal dataset curated to enable fine-grained prediction of both individuallevel behavioral responses and community-wide engagement patterns to health messaging. This dataset supports tasks in multimodal understanding, response prediction, personalization, and social forecasting, allowing rigorous evaluation of how well modern AI systems can emulate, interpret, and anticipate heterogeneous public sentiment and behavior. By providing a new dataset to enable AI advances for public health, PHORECAST aims to catalyze the development of models that are not only more socially aware but also aligned with the goals of adaptive and inclusive health communication

Figures (23)

• Figure 1: Human Nuance vs Model Limitation: Qualitative analysis demonstrates the diverse human reactions and interpretations evoked by a single image, spanning personal concern to broader advocacy. This underscore the significant influence an individual's background and context has on their perception. The right panel shows that current popular models struggle to capture this rich spectrum of human responses and language, often defaulting to repetitive language (e.g., Llama's "I'm not sure..." in 80% of cases). By training with PHORECAST, our models learn to emulate real human language, effectively capturing these subtle distinctions in the data.
• Figure 2: Demographic distribution of participants (N=1095) in our study, showing age groups, gender identity, religion, political affiliation, and highest education attainment. The dataset reflects a broad representation across ages (predominantly 18-44), gender (balanced male/female, inclusive non-binary options), and political views (moderate, liberal, and conservative as most frequent).
• Figure 3: Overview of PHORECAST Pipeline: Via our Survey, we collect human profiles including demographics, personality, locus of control, and opinions on public health topics before and after their interaction with the campaign message. We then train LLM/VLM models to predict different reactions of an individual given a stimuli.
• Figure 4: Differential opinion patterns by education level before and after interacting with stimuli across all topics. Generally, individuals with higher education attainment (Doctoral and Masters; N=183) demonstrate (1) significantly greater concern about different aspects of their health, (2) stronger harm perception of health harms, (3), paradoxically, greater self-reported willingness to engage in harmful behavior such as substance use or smoking. This attitude behavior gap suggests that while higher education enhances risk awareness, it may simultaneously increase behavioral intentions, possibly through increase perceived self behavioral control Assari2019. A detailed demographic and psychographic analysis for each topic is provided in the appendix.
• Figure 5: Example of baseline opinions we ask participants prior to viewing any public health campaigns. Each participants answers four baseline questions on five health topics, such as Chronic Diseases, Substance Use, Smoking/COPD, Nutrition etc