Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MedGPT-oss: Training a General-Purpose Vision-Language Model for Biomedicine

Kai Zhang, Zhengqing Yuan, Cheng Peng, Songlin Zhao, Mengxian Lyu, Ziyi Chen, Yanfang Ye, Wei Liu, Ying Zhang, Kaleb E Smith, Lifang He, Lichao Sun, Yonghui Wu

TL;DR

This work introduces MEDGPT-OSS, an open-weight, 20B-parameter generalist vision-language model designed to facilitate open research in clinical AI, which successfully outperforms larger open medical models on out-of-distribution multimodal reasoning and complex text-only clinical tasks.

Abstract

Biomedical multimodal assistants have the potential to unify radiology, pathology, and clinical-text reasoning, yet a critical deployment gap remains: top-performing systems are either closed-source or computationally prohibitive, precluding the on-premises deployment required for patient privacy and PHI compliance. We introduce MEDGPT-OSS, an open-weight, 20B-parameter generalist vision-language model designed to facilitate open research in clinical AI. Rather than relying on architectural complexity, MEDGPT-OSS pairs the GPT-oss language backbone with a visual front-end via a optimized, three-stage training curriculum. By progressively domain-adapting these modules through rigorous data curation and long-context multimodal alignment, we demonstrate that a 20B model can bridge the capacity gap. It successfully outperforms larger open medical models on out-of-distribution (OOD) multimodal reasoning and complex text-only clinical tasks. By unifying diverse modalities under a single instruction-following interface, MEDGPT-OSS maintains a parameter-efficient footprint fully compatible with commodity GPUs. We release the complete training recipe, open-weight checkpoints, and a rigorous evaluation harness to serve as a verifiable foundation for privacy-preserving, institution-specific clinical AI research.

MedGPT-oss: Training a General-Purpose Vision-Language Model for Biomedicine

TL;DR

This work introduces MEDGPT-OSS, an open-weight, 20B-parameter generalist vision-language model designed to facilitate open research in clinical AI, which successfully outperforms larger open medical models on out-of-distribution multimodal reasoning and complex text-only clinical tasks.

Abstract

Biomedical multimodal assistants have the potential to unify radiology, pathology, and clinical-text reasoning, yet a critical deployment gap remains: top-performing systems are either closed-source or computationally prohibitive, precluding the on-premises deployment required for patient privacy and PHI compliance. We introduce MEDGPT-OSS, an open-weight, 20B-parameter generalist vision-language model designed to facilitate open research in clinical AI. Rather than relying on architectural complexity, MEDGPT-OSS pairs the GPT-oss language backbone with a visual front-end via a optimized, three-stage training curriculum. By progressively domain-adapting these modules through rigorous data curation and long-context multimodal alignment, we demonstrate that a 20B model can bridge the capacity gap. It successfully outperforms larger open medical models on out-of-distribution (OOD) multimodal reasoning and complex text-only clinical tasks. By unifying diverse modalities under a single instruction-following interface, MEDGPT-OSS maintains a parameter-efficient footprint fully compatible with commodity GPUs. We release the complete training recipe, open-weight checkpoints, and a rigorous evaluation harness to serve as a verifiable foundation for privacy-preserving, institution-specific clinical AI research.
Paper Structure (33 sections, 2 figures, 6 tables)

This paper contains 33 sections, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Specialist pipelines and their limits.
  4. Generalist biomedical VLMs and instruction-tuned assistants.
  5. Frontier trends: recipes, long-context grounding, and evaluation.
  6. Positioning of MedGPT-oss.
  7. MedGPT-oss
  8. Model Architecture
  9. Visual Encoder.
  10. Projection Module.
  11. LLM Backbone.
  12. Training Strategies
  13. Pretraining.
  14. Mid-training.
  15. Instruction-tuning.
  16. ...and 18 more sections

Figures (2)

  • Figure 1: Preliminary evaluation of visual encoders on medical multimodal benchmarks. As an initial investigation, we compared the vanilla CLIP backbone against domain-specific alternatives (BiomedCLIP, MedSigLIP) and SigLIP. The models utilize a GPT-oss-20B trained via LoRA (2-stage training following vanilla LLaVA, where we first pretrain the projector, then fine-tune the projector and LLM backbone).
  • Figure 2: Evaluation of multi-view and longitudinal chest X-ray report generation on the MIMIC-CXR benchmark. Performance is measured across three clinically grounded metrics.