Machine Learning for ALSFRS-R Score Prediction: Making Sense of the Sensor Data

TL;DR

The paper investigates predicting ALSFRS-R progression using sensor data from a patient app within the iDPP@CLEF 2024 challenge. It compares a naive baseline, ElasticNet regression, and an LSTM, aided by data augmentation via Task2 and thorough nested grouped cross-validation to handle a small, high-dimensional dataset. A key finding is that the previous ALSFRS-R value is the strongest predictor, with the naive baseline often matching or outperforming more complex models; ElasticNet provides interpretability for feature contributions. The study highlights the potential of sensor data while acknowledging limitations due to dataset size, and it outlines routes for improvement through larger, heterogeneous, and multimodal datasets.

Abstract

Amyotrophic Lateral Sclerosis (ALS) is characterized as a rapidly progressive neurodegenerative disease that presents individuals with limited treatment options in the realm of medical interventions and therapies. The disease showcases a diverse range of onset patterns and progression trajectories, emphasizing the critical importance of early detection of functional decline to enable tailored care strategies and timely therapeutic interventions. The present investigation, spearheaded by the iDPP@CLEF 2024 challenge, focuses on utilizing sensor-derived data obtained through an app. This data is used to construct various machine learning models specifically designed to forecast the advancement of the ALS Functional Rating Scale-Revised (ALSFRS-R) score, leveraging the dataset provided by the organizers. In our analysis, multiple predictive models were evaluated to determine their efficacy in handling ALS sensor data. The temporal aspect of the sensor data was compressed and amalgamated using statistical methods, thereby augmenting the interpretability and applicability of the gathered information for predictive modeling objectives. The models that demonstrated optimal performance were a naive baseline and ElasticNet regression. The naive model achieved a Mean Absolute Error (MAE) of 0.20 and a Root Mean Square Error (RMSE) of 0.49, slightly outperforming the ElasticNet model, which recorded an MAE of 0.22 and an RMSE of 0.50. Our comparative analysis suggests that while the naive approach yielded marginally better predictive accuracy, the ElasticNet model provides a robust framework for understanding feature contributions.

Figures (5)

• Figure 1: Averaged question response values for each follow-up time point. The x-axis shows the ordered visits by patient, whereas the y axis shows the average score. The vertical bars indicate a 95% CI around the estimated mean. As we have fewer and fewer patients for longer follow-ups, the CI widen. There is some degree of variability with respect to the intercept among the questions, as well as the slopes. The beginning periods exhibit slow deterioration which already suggests that using the previous value as predictor for the next score will be useful. The points are jittered for visualization purposes.
• Figure 2: Augmenting Task1's CT scores data with Task2's APP scores. a) An example of patient_id, question_id pair where APP scores are in agreement with CT scores b) An example of patient_id, question_id pair where adding APP scores allows the usability of this datapoint as it had only 1 CT score data c) An example of patient_id, question_id pair where the APP scores aren't in agreement with CT scores.
• Figure 3: Cross-validation procedure for the hyperparameter tuning and final model fit in our work (Source: lyashenko_jha_2024). The original source illustrates the procedure without taking into account a grouped structure as in our dataset. Hence, in our work we adapted the procedure so that each patient's complete data is contained in either the inner-train/test subsets, or the outer test-set.
• Figure 4: Architecture of the LSTM model used for predicting ALSFRS-R scores. The model processes time-series sensor data through sequential LSTM cells, capturing temporal dependencies. The output is combined with baseline scores and static patient data, then fed into multiple linear layers to predict each of the twelve ALSFRS-R sub-scores.
• Figure 5: Feature Importance charts showing "previous_value" is consistently the most important feature.