A General Mixture Loss Function to Optimize a Personalized Predictive Model

TL;DR

This paper addresses personalization in health predictions by optimizing the subpopulation size $M$ used to train individualized predictive models. It introduces a generalized mixture loss $L^{(M)}$ that jointly balances discrimination and calibration through a mixture weight $\alpha$ and user-specified measures, with optional LOESS smoothing for calibration. Through extensive simulations, it shows that the optimal $M$ depends on the chosen loss terms and that bounding the grid of $M$ (e.g., 20%–70% of the training data) substantially reduces computation while preserving performance; the results are reinforced with a real-data analysis on the eICU mortality dataset. The findings support a flexible, computation-aware approach to deploying personalized predictive models in precision health, with guidance on how the calibration/discrimination trade-off shapes subpopulation size and performance.

Abstract

Advances in precision medicine increasingly drive methodological innovation in health research. A key development is the use of personalized prediction models (PPMs), which are fit using a similar subpopulation tailored to a specific index patient, and have been shown to outperform one-size-fits-all models, particularly in terms of model discrimination performance. We propose a generalized loss function that enables tuning of the subpopulation size used to fit a PPM. This loss function allows joint optimization of discrimination and calibration, allowing both the performance measures and their relative weights to be specified by the user. To reduce computational burden, we conducted extensive simulation studies to identify practical bounds for the grid of subpopulation sizes. Based on these results, we recommend using a lower bound of 20\% and an upper bound of 70\% of the entire training dataset. We apply the proposed method to both simulated and real-world datasets and demonstrate that previously observed relationships between subpopulation size and model performance are robust. Furthermore, we show that the choice of performance measures in the loss function influences the optimal subpopulation size selected. These findings support the flexible and computationally efficient implementation of PPMs in precision health research.

Figures (5)

• Figure 1: Performance measures as a function of $M$ for Dataset 1 from Table \ref{['table: datasets']}.
• Figure 2: Performance measures as a function of $M$ for Dataset 2 from Table \ref{['table: datasets']}.
• Figure 3: Performance measures as a function of $M$ for Datasets 1 and 2 from Table \ref{['table: datasets']}.
• Figure 4: Performance measures as a function of $M$ for the cardiac dataset from the eICU database.
• Figure 5: Optimal $M$ proportions found using $L^*$ from Equation (5), and $L^{**}$ from Equation (8), for varying values of $\alpha$ found when tuning $M$ during the training/testing step of applying the proposed algorithm to the eICU dataset.