Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Truth, Trust, and Trouble: Medical AI on the Edge

Mohammad Anas Azeez, Rafiq Ali, Ebad Shabbir, Zohaib Hasan Siddiqui, Gautam Siddharth Kashyap, Jiechao Gao, Usman Naseem

TL;DR

This work systematically benchmarks open-source medical LLMs (AlpaCare-13B, BioMistral-7B-DARE, Mistral-7B) on long-form medical Q&A using a 1,077-question anatomy benchmark created from textbooks and de-identified clinical notes, with double-blind A/B testing and expert physician annotations. It demonstrates that domain-specific tuning enhances factual accuracy, with AlpaCare-13B achieving about 91.7% accuracy and 0.92 harmlessness, while BioMistral-7B-DARE achieves strong safety despite a smaller size. Few-shot prompting improves accuracy from 78% to 85% and boosts honesty and helpfulness, but all models struggle with complex reasoning and edge-case generalization remains a risk. The study provides industry-ready guidance for deploying open-source clinical LLMs, emphasizing layered safety, structured evaluation, and human oversight in regulated healthcare settings.

Abstract

Large Language Models (LLMs) hold significant promise for transforming digital health by enabling automated medical question answering. However, ensuring these models meet critical industry standards for factual accuracy, usefulness, and safety remains a challenge, especially for open-source solutions. We present a rigorous benchmarking framework using a dataset of over 1,000 health questions. We assess model performance across honesty, helpfulness, and harmlessness. Our results highlight trade-offs between factual reliability and safety among evaluated models -- Mistral-7B, BioMistral-7B-DARE, and AlpaCare-13B. AlpaCare-13B achieves the highest accuracy (91.7%) and harmlessness (0.92), while domain-specific tuning in BioMistral-7B-DARE boosts safety (0.90) despite its smaller scale. Few-shot prompting improves accuracy from 78% to 85%, and all models show reduced helpfulness on complex queries, highlighting ongoing challenges in clinical QA.

Truth, Trust, and Trouble: Medical AI on the Edge

TL;DR

This work systematically benchmarks open-source medical LLMs (AlpaCare-13B, BioMistral-7B-DARE, Mistral-7B) on long-form medical Q&A using a 1,077-question anatomy benchmark created from textbooks and de-identified clinical notes, with double-blind A/B testing and expert physician annotations. It demonstrates that domain-specific tuning enhances factual accuracy, with AlpaCare-13B achieving about 91.7% accuracy and 0.92 harmlessness, while BioMistral-7B-DARE achieves strong safety despite a smaller size. Few-shot prompting improves accuracy from 78% to 85% and boosts honesty and helpfulness, but all models struggle with complex reasoning and edge-case generalization remains a risk. The study provides industry-ready guidance for deploying open-source clinical LLMs, emphasizing layered safety, structured evaluation, and human oversight in regulated healthcare settings.

Abstract

Large Language Models (LLMs) hold significant promise for transforming digital health by enabling automated medical question answering. However, ensuring these models meet critical industry standards for factual accuracy, usefulness, and safety remains a challenge, especially for open-source solutions. We present a rigorous benchmarking framework using a dataset of over 1,000 health questions. We assess model performance across honesty, helpfulness, and harmlessness. Our results highlight trade-offs between factual reliability and safety among evaluated models -- Mistral-7B, BioMistral-7B-DARE, and AlpaCare-13B. AlpaCare-13B achieves the highest accuracy (91.7%) and harmlessness (0.92), while domain-specific tuning in BioMistral-7B-DARE boosts safety (0.90) despite its smaller scale. Few-shot prompting improves accuracy from 78% to 85%, and all models show reduced helpfulness on complex queries, highlighting ongoing challenges in clinical QA.

Paper Structure

This paper contains 22 sections, 3 equations, 2 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Data Collection and QA Generation
  5. Annotations:
  6. Benchmarking Protocol
  7. Experimental Setup
  8. Evaluation Metrics
  9. Honesty.
  10. Helpfulness.
  11. Harmlessness.
  12. Hyperparameters
  13. Experimental Analysis
  14. Accuracy vs. Specialization
  15. Model Scale vs. Safety
  16. ...and 7 more sections

Figures (2)

  • Figure 1: Examples of QA generation and filtering across rule-based, LLM-generated, and edge-case filtered methods.
  • Figure 2: Semantic and temporal analysis of question-answer behavior. (a) t-SNE shows semantic clustering with difficulty overlay. (b) Heatmap illustrates lexical distribution across question indices. (c) Rolling correctness vs. difficulty trends. (d–e) Word clouds highlight frequent terms in TRUE and FALSE answers. (f) Evolution of answer types over time.