Counterfactual Modeling with Fine-Tuned LLMs for Health Intervention Design and Sensor Data Augmentation

TL;DR

Counterfactual explanations provide actionable recourse and interpretability. The paper introduces SenseCF, fine-tuning LLMs to generate CFs for intervention design and data augmentation on sensor-based health data. It systematically compares GPT-4o, BioMistral-7B, and LLaMA-3.1-8B in zero-shot and fine-tuned modes using the AI-READI dataset, reporting high plausibility and substantial F1 recovery under label scarcity, especially with fine-tuned LLaMA. The findings support a model-agnostic CF generation approach that yields clinically actionable adjustments and improves data efficiency in digital health pipelines.

Abstract

Counterfactual explanations (CFEs) provide human-centric interpretability by identifying the minimal, actionable changes required to alter a machine learning model's prediction. Therefore, CFs can be used as (i) interventions for abnormality prevention and (ii) augmented data for training robust models. We conduct a comprehensive evaluation of CF generation using large language models (LLMs), including GPT-4 (zero-shot and few-shot) and two open-source models-BioMistral-7B and LLaMA-3.1-8B, in both pretrained and fine-tuned configurations. Using the multimodal AI-READI clinical dataset, we assess CFs across three dimensions: intervention quality, feature diversity, and augmentation effectiveness. Fine-tuned LLMs, particularly LLaMA-3.1-8B, produce CFs with high plausibility (up to 99%), strong validity (up to 0.99), and realistic, behaviorally modifiable feature adjustments. When used for data augmentation under controlled label-scarcity settings, LLM-generated CFs substantially restore classifier performance, yielding an average 20% F1 recovery across three scarcity scenarios. Compared with optimization-based baselines such as DiCE, CFNOW, and NICE, LLMs offer a flexible, model-agnostic approach that generates more clinically actionable and semantically coherent counterfactuals. Overall, this work demonstrates the promise of LLM-driven counterfactuals for both interpretable intervention design and data-efficient model training in sensor-based digital health. Impact: SenseCF fine-tunes an LLM to generate valid, representative counterfactual explanations and supplement minority class in an imbalanced dataset for improving model training and boosting model robustness and predictive performance

Figures (8)

• Figure 1: F1-Score Decline Across Models as Training Data is Reduced.
• Figure 2: SenseCF pipeline: LLM-generated counterfactuals are used both for augmenting imbalanced training data (left) and for model interpretability (right).
• Figure 3: Counterfactual generation using LLMs from sensor-derived features.
• Figure 4: Prompt template for counterfactual generation.
• Figure 5: Overview of the feature extraction pipeline. Patients are first split into train and test sets to avoid data leakage. A two-hour sliding window with 50% overlap is applied to continuous signals, followed by extraction of sleep, CGM, activity, and demographic features.