Improving the Prediction of Individual Engagement in Recommendations Using Cognitive Models

TL;DR

The paper addresses predicting individual engagement in health-promotion programs under resource constraints, modeling non-Markovian dynamics with personalized Instance-Based Learning (IBL) instead of purely data-driven forecasters. By training per-beneficiary IBL models and using their blended value to predict next-period engagement, the approach yields more accurate one-step forecasts than LSTMs and reveals interpretable individual-difference profiles. These cognitive models also enable clustering of participants and show how cluster-informed training can boost performance of other time-series models; results suggest practical benefits for targeting interventions and understanding behavior. Overall, the work demonstrates that cognitively grounded memory and similarity mechanisms can improve engagement predictions and inform more efficient, personalized intervention strategies in real-world health programs.

Abstract

For public health programs with limited resources, the ability to predict how behaviors change over time and in response to interventions is crucial for deciding when and to whom interventions should be allocated. Using data from a real-world maternal health program, we demonstrate how a cognitive model based on Instance-Based Learning (IBL) Theory can augment existing purely computational approaches. Our findings show that, compared to general time-series forecasters (e.g., LSTMs), IBL models, which reflect human decision-making processes, better predict the dynamics of individuals' states. Additionally, IBL provides estimates of the volatility in individuals' states and their sensitivity to interventions, which can improve the efficiency of training of other time series models.

Figures (7)

• Figure 1: a) Transforming a beneficiary's trajectory into IBL instances and LSTM sliding windows. Each IBL instance represents the engagement context at a given timestep and consists of three parts: the engagement level on the immediately prior timestep, the number of timesteps since the last intervention, and the engagement level on the current timestep. In contrast, the LSTM data was constructed by segmenting trajectories into sliding windows of length 7. b) Training / testing setup for comparing IBL and LSTM methods. Different colors indicate data from individual beneficiaries. Under the IBL procedure, one IBL model is trained per beneficiary. Each trained IBL model is then used to predict their respective beneficiaries during testing. Under the LSTM procedure, one LSTM is trained using data from all beneficiaries. The trained LSTM model is then used to make predictions for all beneficiaries during testing. c) LSTM training / testing setups under various IBL-clusters-related conditions. Different colors indicate different cluster types. Note that the LSTM in the Random condition is trained on a random subset of one-third of the training data as compared to the other training conditions.
• Figure 2: Next-step prediction error per testing time step (smaller is better). N = 210. The personalized IBL models consistently achieves lower average prediction error (0.23 across all time steps) as compared to the LSTM model (0.32), which translates to a 29% reduction in error.
• Figure 3: Distribution of attribute weight profiles. Each dot represents the IBL model personalized to a particular beneficiary (mother). Points are slightly jittered to better visualize density. Lighter points indicate training set mothers while darker points indicate testing set mothers. Beneficiaries are clustered into 3 clusters according to a k-means algorithm. Larger circles outlined in black indicate cluster centroids.
• Figure 4: Next-step prediction error per testing time step for the various LSTM training-testing methods. Note that the performance for the Outside-cluster method is an average of the performance of two LSTM models (e.g., for a state-stable mother, the average prediction error of a LSTM trained on intervention-sensitive mothers and a LSTM trained on transition-consistent mothers.
• Figure 5: Comparison of prediction errors per testing mother between Entire and Within-cluster methods. Blue lines indicate mothers who are (on average) better predicted by the Within-cluster method whereas red lines indicate those better predicted by the Entire method.