Exploring General-Purpose Autonomous Multimodal Agents for Pathology Report Generation
Marc Aubreville, Taryn A. Donovan, Christof A. Bertram
TL;DR
This study assesses whether general-purpose agentic vision–language systems can autonomously navigate digital slides, describe histopathology, and provide provisional diagnoses in veterinary cases. Using 35 whole-slide images across three conditioning scenarios and two agentic frameworks, the authors evaluate diagnostic accuracy against board-certified pathologists. Results reveal sizable gaps in AI performance, with accuracy climbing when morphological descriptions are provided but remaining well below human expert levels, highlighting issues of hallucination and incomplete targeted inspection. The work establishes a benchmark for agentic vision–language systems in a domain-specific, high-stakes setting and underscores the need for cautious application and further domain-tailored development.
Abstract
Recent advances in agentic artificial intelligence, i.e. systems capable of autonomous perception, reasoning, and tool use, offer new opportunities for digital pathology. In this pilot study, we evaluate whether two agentic multimodal AI systems (OpenAI's ChatGPT 5.0 in agentic mode, and H Company's Surfer) can autonomously navigate, describe, and interpret histopathologic features in digitized tissue slides on a slide viewing platform. A set of 35 veterinary pathology cases, curated for training purposes, was used as the test dataset. The agent was tasked with autonomously exploring whole-slide images using a web-based slide viewer, identifying salient tissue structures, generating descriptive summaries, and proposing provisional diagnoses. We fed different prompts to explore three scenarios: 1) analysis without knowledge of the signalment, 2) analysis with organ and species provided, and 3) diagnosis based on a morphological description provided. All outputs were reviewed and validated by a board-certified pathologist for accuracy and diagnostic consistency. We further tasked another board-certified pathologist with the same task to establish a baseline. We found the systems to yield accurate diagnoses in up to 28.6% of cases with only images, signalment and organ provided, and up to 68.6% when a morphological description was provided. With only the WSI provided, the models were only correct in up to 5.7% of cases. The human expert, on the other hand, achieved 85.7% diagnostic accuracy with only a single WSI, and 88.6% when also signalment and organ was provided. The study demonstrates that while the agentic AI system can meaningfully engage with web-based slide viewing software to assess complex visual pathology data and produce contextually aligned feature descriptions, diagnostic precision remains limited compared with a human expert.