Bespoke Large Language Models for Digital Triage Assistance in Mental Health Care

TL;DR

This work addresses the challenge of triaging NHS mental health referrals by leveraging bespoke, end-to-end LLM-based ingestion of unstructured EHR notes. It evaluates multiple strategies for handling variable-length clinical narratives, revealing that a segment-and-batch approach with a RoBERTa-OHFT encoder offers the best trade-off between performance and resource use on a single GPU. By using a 14-day acceptance heuristic and focusing on five sub-specialty teams, the study demonstrates strong classification performance while highlighting the importance of explainability and governance in clinical AI. The findings have practical significance for reducing referral delays and increasing transparency in triage decisions within resource-constrained NHS settings.

Abstract

Contemporary large language models (LLMs) may have utility for processing unstructured, narrative free-text clinical data contained in electronic health records (EHRs) -- a particularly important use-case for mental health where a majority of routinely-collected patient data lacks structured, machine-readable content. A significant problem for the the United Kingdom's National Health Service (NHS) are the long waiting lists for specialist mental healthcare. According to NHS data, in each month of 2023, there were between 370,000 and 470,000 individual new referrals into secondary mental healthcare services. Referrals must be triaged by clinicians, using clinical information contained in the patient's EHR to arrive at a decision about the most appropriate mental healthcare team to assess and potentially treat these patients. The ability to efficiently recommend a relevant team by ingesting potentially voluminous clinical notes could help services both reduce referral waiting times and with the right technology, improve the evidence available to justify triage decisions. We present and evaluate three different approaches for LLM-based, end-to-end ingestion of variable-length clinical EHR data to assist clinicians when triaging referrals. Our model is able to deliver triage recommendations consistent with existing clinical practices and it's architecture was implemented on a single GPU, making it practical for implementation in resource-limited NHS environments where private implementations of LLM technology will be necessary to ensure confidential clinical data is appropriately controlled and governed.

Figures (10)

• Figure 1: A: Schematic description of the triage process using referral and historical EHR documents highlighting the accept/reject and sub-specialty referral and re-triage process; SPoA = single-point-of-access refers to the common location that receives all referrals and where a number of clinicians will perform triage B: End-to-end ingestion of the same clinical data for assisted triage. Example sub-specialty teams: EIP = early intervention for psychosis, ED = eating disorders, ID = intellectual disability
• Figure 2: See main text for description of each approach
• Figure 3: Validation set F1 score for each method of handling variable sequence lengths as a function of instance token lengths: short ($<128$ tokens), medium ($>128$ and $<=512$ tokens), long ($>512$ and $<=4096$ tokens), and extra long ($>4096$ tokens). For details of each method, refer back to Fig \ref{['fig:three-methods']} where Brute refers to method A, Concat trunc. and Concat Long. refer to B using standard RoBERTa architectures and the longformer respectively and Segment-batch to method C.
• Figure 4: Tabulation of the probability of the team first receiving a referral (Team A, rows) referring the patient onto another team (team B, columns) within a 30 day window.
• Figure 5: Histogram of the number of days individual referral instances remain open based on the available referral date, and discharge date structured fields within the OHFT EHR data.