LLaDA-MedV: Exploring Large Language Diffusion Models for Biomedical Image Understanding

TL;DR

A detailed comparison with LLaVA-Med suggests that LLaDA-MedV is capable of generating reasonably longer responses by explicitly controlling response length, which can lead to more informative outputs.

Abstract

Autoregressive models (ARMs) have long dominated the landscape of biomedical vision-language models (VLMs). Recently, masked diffusion models such as LLaDA have emerged as promising alternatives, yet their application in the biomedical domain remains largely underexplored. To bridge this gap, we introduce LLaDA-MedV, the first large language diffusion model tailored for biomedical image understanding through vision instruction tuning. LLaDA-MedV achieves relative performance gains of 7.855% over LLaVA-Med and 1.867% over LLaDA-V in the open-ended biomedical visual conversation task, and sets new state-of-the-art accuracy on the closed-form subset of three VQA benchmarks: 84.93% on VQA-RAD, 92.31% on SLAKE, and 95.15% on PathVQA. Furthermore, a detailed comparison with LLaVA-Med suggests that LLaDA-MedV is capable of generating reasonably longer responses by explicitly controlling response length, which can lead to more informative outputs. We also conduct an in-depth analysis of both the training and inference stages, highlighting the critical roles of initialization weight selection, fine-tuning strategies, and the interplay between sampling steps and response repetition. The code and model weight is released at https://github.com/LLM-VLM-GSL/LLaDA-MedV.

Figures (12)

• Figure 1: Illustration of biomedical VLMs evaluated in the open-ended biomedical conversation benchmark. Among the 6 Medical VLMs, LLaDA-MedV achieves the highest overall score and demonstrates the best performance on Chest X-ray (CXR) and CT modalities.
• Figure 2: Illustration of open-end conversation evaluation. All questions, images and corresponding captions are sourced from li2023llava. We present representative responses from (a) LLaVA-Med, (b) MedVLM-R1, and (c) LLaDA-MedV. Key informative segments are highlighted in red for emphasis.
• Figure 3: Illustration of LLaDA-MedV responses on downstream VQA tasks across three benchmarks. (A) and (B) represent the open-form QA examples from VQA-RAD and SLAKE benchmarks, respectively. (C) represents the closed-form (i.e., yes/no) QA from PathVQA benchmark. For this experiment, we set $L = B = Z = 64$.
• Figure 4: Illustration of LLaVA-Med and LLaDA-MedV responses to biomedical queries 1 and 2. The images, queries, and corresponding captions are adapted from li2023llava. For fair comparison, the generation lengths are aligned by setting the maximum token limit to the same value (e.g., generation length $L = 256$).
• Figure 5: Illustration of token repetition during generation (i.e., mark by red) across different settings. Question 4 and 5 represernt the answer from LLaDA-MedV$^{V_1}$ and LLaDA-MedV$^{V_2}$, respectively. We omit the corresponding images for clarity.