Artificial intelligence to improve clinical coding practice in Scandinavia: a crossover randomized controlled trial

TL;DR

The potential of AI to transform common tasks in clinical workflows, with ostensible positive impacts on work efficiencies for clinical coding tasks with more demanding longer text sequences, is demonstrated.

Abstract

\textbf{Trial design} Crossover randomized controlled trial. \textbf{Methods} An AI tool, Easy-ICD, was developed to assist clinical coders and was tested for improving both accuracy and time in a user study in Norway and Sweden. Participants were randomly assigned to two groups, and crossed over between coding complex (longer) texts versus simple (shorter) texts, while using our tool versus not using our tool. \textbf{Results} Based on Mann-Whitney U test, the median coding time difference for complex clinical text sequences was 123 seconds (\emph{P}\textless.001, 95\% CI: 81 to 164), representing a 46\% reduction in median coding time when our tool is used. There was no significant time difference for simpler text sequences. For coding accuracy, the improvement we noted for both complex and simple texts was not significant. \textbf{Conclusions} This study demonstrates the potential of AI to transform common tasks in clinical workflows, with ostensible positive impacts on work efficiencies for complex clinical coding tasks. Further studies within hospital workflows are required before these presumed impacts can be more clearly understood.

Figures (5)

• Figure 1: Crossover study design illustrating the AB$|$BA sequences, where Period 1 contains 65% of the total word count and Period 2 contains 35% of the total word count.
• Figure 2: Screen dump of the Easy-ICD DEMO.
• Figure 3: Crossover study design illustrating the AB$|$BA sequences, where Period 1 contains 10 complex clinical notes and Period 2 has an additional 10 simple clinical notes, for a total of 300 observations in this study.
• Figure 4: Clinical coding time (sec.) for complex and simple clinical texts.
• Figure 5: Qualitative feedback in the form of 5-star rating for the suggested codes for complex notes (top) and simple notes (bottom) for the 55 submitted ratings out of a possible total of 150 notes that could have been rated.