Leveraging Machine Learning Models to Predict the Outcome of Digital Medical Triage Interviews

TL;DR

The paper tackles predicting triage outcomes from unfinished digital interviews by framing it as a multiclass classification on sparse, high-dimensional questionnaire data from Platform24's Triage24. It compares five tree-based models and the TabTransformer, finding that LGBMClassifier and CatBoostClassifier achieve robust accuracy (>80%) on complete interviews and retain strong performance as interviews become incomplete, with TabTransformer offering stable accuracy at the cost of long training times. A key result is the linear relationship between interview completeness and predictive power for most models, while TabTransformer remains resilient to missing data. The work demonstrates practical feasibility for integrating an ML side-car into deterministic digital triage to aid patients who exit interviews prematurely and highlights concrete directions for scalable, interpretable, and resource-aware deployment in healthcare settings.

Abstract

Many existing digital triage systems are questionnaire-based, guiding patients to appropriate care levels based on information (e.g., symptoms, medical history, and urgency) provided by the patients answering questionnaires. Such a system often uses a deterministic model with predefined rules to determine care levels. It faces challenges with incomplete triage interviews since it can only assist patients who finish the process. In this study, we explore the use of machine learning (ML) to predict outcomes of unfinished interviews, aiming to enhance patient care and service quality. Predicting triage outcomes from incomplete data is crucial for patient safety and healthcare efficiency. Our findings show that decision-tree models, particularly LGBMClassifier and CatBoostClassifier, achieve over 80\% accuracy in predicting outcomes from complete interviews while having a linear correlation between the prediction accuracy and interview completeness degree. For example, LGBMClassifier achieves 88,2\% prediction accuracy for interviews with 100\% completeness, 79,6\% accuracy for interviews with 80\% completeness, 58,9\% accuracy for 60\% completeness, and 45,7\% accuracy for 40\% completeness. The TabTransformer model demonstrated exceptional accuracy of over 80\% for all degrees of completeness but required extensive training time, indicating a need for more powerful computational resources. The study highlights the linear correlation between interview completeness and predictive power of the decision-tree models.

Figures (4)

• Figure 1: System integration of the ML model into the existing triage
• Figure 2: An example of a Triage24 interview (in Swedish) with the immediate urgency outcome. English: 1. Do you have a headache right now?: Yes/No. 2. Did your headache start in connection with any of the following?: Cold or flu/Blows or other violence to the head/None of the above. 3. In what way have you injured your head? (Choose all that apply.): Injury to the head/Hit your head on or against something. 4. Was it a serious injury or impact with great force? (E.g., fall over 5 meters, hit by a car, car accident at more than 50 km/h, motorcycle accident at more than 35 km/h): Yes/No. 5. Thank you for your answers! You have suffered a blow or injury that has potentially put a lot of strain on your body. You can now review your answers and others if necessary before continuing. 6. You should be assessed immediately. You should call 112.
• Figure 3: Two patient interviews with different outcomes based on the questionnaires
• Figure 4: Correlation between completeness level and prediction accuracy