Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeepScore: A Comprehensive Approach to Measuring Quality in AI-Generated Clinical Documentation

Jon Oleson

TL;DR

An overview of DeepScribe's methodologies for assessing and managing note quality, focusing on various metrics and the composite "DeepScore", an overall index of quality and accuracy are presented.

Abstract

Medical practitioners are rapidly adopting generative AI solutions for clinical documentation, leading to significant time savings and reduced stress. However, evaluating the quality of AI-generated documentation is a complex and ongoing challenge. This paper presents an overview of DeepScribe's methodologies for assessing and managing note quality, focusing on various metrics and the composite "DeepScore", an overall index of quality and accuracy. These methodologies aim to enhance the quality of patient care documentation through accountability and continuous improvement.

DeepScore: A Comprehensive Approach to Measuring Quality in AI-Generated Clinical Documentation

TL;DR

An overview of DeepScribe's methodologies for assessing and managing note quality, focusing on various metrics and the composite "DeepScore", an overall index of quality and accuracy are presented.

Abstract

Medical practitioners are rapidly adopting generative AI solutions for clinical documentation, leading to significant time savings and reduced stress. However, evaluating the quality of AI-generated documentation is a complex and ongoing challenge. This paper presents an overview of DeepScribe's methodologies for assessing and managing note quality, focusing on various metrics and the composite "DeepScore", an overall index of quality and accuracy. These methodologies aim to enhance the quality of patient care documentation through accountability and continuous improvement.
Paper Structure (27 sections, 3 equations, 5 figures, 6 tables)

This paper contains 27 sections, 3 equations, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Statistical Rates
  4. Results
  5. Overview
  6. Key Concepts
  7. Metric Definitions
  8. Recall to Precision Funnel
  9. Results
  10. Metric Definitions
  11. User Acceptance
  12. Results
  13. Key Concepts
  14. Metric Definitions
  15. Transcription Quality Control
  16. ...and 12 more sections

Figures (5)

  • Figure 1: A rubric with highlighted entities.
  • Figure 2: A test note (left) with snippets connected to rubric entities (right).
  • Figure 3: A demonstration of how CER is calculated on a single note.
  • Figure 4: A demonstration of how AER is calculated on the same note, with reference to the rubric.
  • Figure 5: Pct Notes in Words Substituted Rate Segment