Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Automatic Mapping of Anatomical Landmarks from Free-Text Using Large Language Models: Insights from Llama-2

Mohamad Abdi, Gerardo Hermosillo Valadez, Halid Ziya Yerebakan

TL;DR

Through experiments with Llama-2 models, it is found that they can linearly represent anatomical landmarks in space with considerable robustness to different prompts, and underscore the potential of LLMs to enhance the efficiency and accuracy of medical imaging workflows.

Abstract

Anatomical landmarks are vital in medical imaging for navigation and anomaly detection. Modern large language models (LLMs), like Llama-2, offer promise for automating the mapping of these landmarks in free-text radiology reports to corresponding positions in image data. Recent studies propose LLMs may develop coherent representations of generative processes. Motivated by these insights, we investigated whether LLMs accurately represent the spatial positions of anatomical landmarks. Through experiments with Llama-2 models, we found that they can linearly represent anatomical landmarks in space with considerable robustness to different prompts. These results underscore the potential of LLMs to enhance the efficiency and accuracy of medical imaging workflows.

Automatic Mapping of Anatomical Landmarks from Free-Text Using Large Language Models: Insights from Llama-2

TL;DR

Through experiments with Llama-2 models, it is found that they can linearly represent anatomical landmarks in space with considerable robustness to different prompts, and underscore the potential of LLMs to enhance the efficiency and accuracy of medical imaging workflows.

Abstract

Anatomical landmarks are vital in medical imaging for navigation and anomaly detection. Modern large language models (LLMs), like Llama-2, offer promise for automating the mapping of these landmarks in free-text radiology reports to corresponding positions in image data. Recent studies propose LLMs may develop coherent representations of generative processes. Motivated by these insights, we investigated whether LLMs accurately represent the spatial positions of anatomical landmarks. Through experiments with Llama-2 models, we found that they can linearly represent anatomical landmarks in space with considerable robustness to different prompts. These results underscore the potential of LLMs to enhance the efficiency and accuracy of medical imaging workflows.

Paper Structure

This paper contains 12 sections, 3 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Representation Engineering
  4. LLMs as world models
  5. Methods
  6. Datasets
  7. Llama-2 Encoding
  8. Probing
  9. Experiments and evaluations
  10. Results
  11. Conclusion
  12. Limitations

Figures (1)

  • Figure 1: Mean Euclidean distance as a function of model depth for the test set. The predicted positions obtained using the linear and nonlinear (MLP) probes were compared to their corresponding values. Each probe was trained and applied to the random prompt (random), context-inducing prompt (prompt), and the unprompted (empty) activation datasets.