Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Long Context: When Semantics Matter More than Tokens

Tarun Kumar Chawdhury, Jon D. Duke

TL;DR

This work evaluates the Clinical Entity Augmented Retrieval (CLEAR) approach for semantic question answering over long, unstructured EHR notes embedded in FHIR DocumentReference, offering a platform to benchmark CLEAR against zero-shot long-context and embedding-based RAG baselines. The Enhanced CLEAR pipeline introduces keyword-based entity extraction, section-aware processing, a targeted context selection strategy, and token optimization to improve clinical reasoning while reducing token usage. On synthetic, de-identified notes spanning 10k–65k tokens, CLEAR achieves higher semantic similarity (avg ~0.878) and a 58.3% win rate, with substantial token savings (~78%) versus wide-context processing, and shows strong scalability for very long documents. The result is a reproducible evaluation framework that supports robust, cost-aware clinical QA research and informs deployment decisions in resource-constrained healthcare environments.

Abstract

Electronic Health Records (EHR) store clinical documentation as base64 encoded attachments in FHIR DocumentReference resources, which makes semantic question answering difficult. Traditional vector database methods often miss nuanced clinical relationships. The Clinical Entity Augmented Retrieval (CLEAR) method, introduced by Lopez et al. 2025, uses entity aware retrieval and achieved improved performance with an F1 score of 0.90 versus 0.86 for embedding based retrieval, while using over 70 percent fewer tokens. We developed a Clinical Notes QA Evaluation Platform to validate CLEAR against zero shot large context inference and traditional chunk based retrieval augmented generation. The platform was tested on 12 clinical notes ranging from 10,000 to 65,000 tokens representing realistic EHR content. CLEAR achieved a 58.3 percent win rate, an average semantic similarity of 0.878, and used 78 percent fewer tokens than wide context processing. The largest performance gains occurred on long notes, with a 75 percent win rate for documents exceeding 65,000 tokens. These findings confirm that entity aware retrieval improves both efficiency and accuracy in clinical natural language processing. The evaluation framework provides a reusable and transparent benchmark for assessing clinical question answering systems where semantic precision and computational efficiency are critical.

Beyond Long Context: When Semantics Matter More than Tokens

TL;DR

This work evaluates the Clinical Entity Augmented Retrieval (CLEAR) approach for semantic question answering over long, unstructured EHR notes embedded in FHIR DocumentReference, offering a platform to benchmark CLEAR against zero-shot long-context and embedding-based RAG baselines. The Enhanced CLEAR pipeline introduces keyword-based entity extraction, section-aware processing, a targeted context selection strategy, and token optimization to improve clinical reasoning while reducing token usage. On synthetic, de-identified notes spanning 10k–65k tokens, CLEAR achieves higher semantic similarity (avg ~0.878) and a 58.3% win rate, with substantial token savings (~78%) versus wide-context processing, and shows strong scalability for very long documents. The result is a reproducible evaluation framework that supports robust, cost-aware clinical QA research and informs deployment decisions in resource-constrained healthcare environments.

Abstract

Electronic Health Records (EHR) store clinical documentation as base64 encoded attachments in FHIR DocumentReference resources, which makes semantic question answering difficult. Traditional vector database methods often miss nuanced clinical relationships. The Clinical Entity Augmented Retrieval (CLEAR) method, introduced by Lopez et al. 2025, uses entity aware retrieval and achieved improved performance with an F1 score of 0.90 versus 0.86 for embedding based retrieval, while using over 70 percent fewer tokens. We developed a Clinical Notes QA Evaluation Platform to validate CLEAR against zero shot large context inference and traditional chunk based retrieval augmented generation. The platform was tested on 12 clinical notes ranging from 10,000 to 65,000 tokens representing realistic EHR content. CLEAR achieved a 58.3 percent win rate, an average semantic similarity of 0.878, and used 78 percent fewer tokens than wide context processing. The largest performance gains occurred on long notes, with a 75 percent win rate for documents exceeding 65,000 tokens. These findings confirm that entity aware retrieval improves both efficiency and accuracy in clinical natural language processing. The evaluation framework provides a reusable and transparent benchmark for assessing clinical question answering systems where semantic precision and computational efficiency are critical.

Paper Structure

This paper contains 28 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Protocol Alignment with CLEAR and Key Differences
  5. Enhanced CLEAR Implementation
  6. Enhanced Entity Extraction
  7. Section-Aware Processing
  8. Context Selection Algorithm
  9. Baseline Methods
  10. Evaluation Framework
  11. Evaluation Application and Cross-Model Protocol
  12. Dataset Generation and Baseline Construction
  13. Results
  14. Overall Performance Comparison
  15. Detailed Performance Analysis
  16. ...and 13 more sections

Figures (4)

  • Figure 1: Baseline performance comparison showing distribution of best strategies across notes before efficiency adjustment.
  • Figure 2: CLEAR dominates at a 3% efficiency bonus, maintaining superior adjusted accuracy across all notes.
  • Figure 3: RAG becomes optimal only under a 14% efficiency bonus, reflecting trade-offs between token cost and accuracy.
  • Figure 4: Interactive interface showing different analysis approaches and their corresponding performance scores, enabling experimentation to improve reasoning accuracy.