Standardization of Psychiatric Diagnoses -- Role of Fine-tuned LLM Consortium and OpenAI-gpt-oss Reasoning LLM Enabled Decision Support System

TL;DR

Psychiatric diagnoses are highly subjective due to reliance on clinician–patient dialogues, leading to variability in outcomes. The paper introduces a four-layer AI-assisted diagnostic framework that ensembles fine-tuned LLMs (e.g., Llama-3, Mistral, Qwen2) and a reasoning LLM (OpenAI-gpt-oss) to generate DSM-5–aligned diagnoses, orchestrated by dedicated AI agents and data stored in a DSM-5–annotated Data Lake. Key contributions include end-to-end platform design, LoRA/QLoRA-efficient fine-tuning on consumer hardware, and a consensus-based reasoning layer that improves robustness and interpretability. The prototype, developed with collaborators from the U.S. Army Medical Research Team, demonstrates potential for standardized, transparent, and scalable AI-assisted psychiatric decision support in real-world eHealth settings.

Abstract

The diagnosis of most mental disorders, including psychiatric evaluations, primarily depends on dialogues between psychiatrists and patients. This subjective process can lead to variability in diagnoses across clinicians and patients, resulting in inconsistencies and challenges in achieving reliable outcomes. To address these issues and standardize psychiatric diagnoses, we propose a Fine-Tuned Large Language Model (LLM) Consortium and OpenAI-gpt-oss Reasoning LLM-enabled Decision Support System for the clinical diagnosis of mental disorders. Our approach leverages fine-tuned LLMs trained on conversational datasets involving psychiatrist-patient interactions focused on mental health conditions (e.g., depression). The diagnostic predictions from individual models are aggregated through a consensus-based decision-making process, refined by the OpenAI-gpt-oss reasoning LLM. We propose a novel method for deploying LLM agents that orchestrate communication between the LLM consortium and the reasoning LLM, ensuring transparency, reliability, and responsible AI across the entire diagnostic workflow. Experimental results demonstrate the transformative potential of combining fine-tuned LLMs with a reasoning model to create a robust and highly accurate diagnostic system for mental health assessment. A prototype of the proposed platform, integrating three fine-tuned LLMs with the OpenAI-gpt-oss reasoning LLM, was developed in collaboration with the U.S. Army Medical Research Team in Norfolk, Virginia, USA. To the best of our knowledge, this work represents the first application of a fine-tuned LLM consortium integrated with a reasoning LLM for clinical mental health diagnosis paving the way for next-generation AI-powered eHealth systems aimed at standardizing psychiatric diagnoses.

Figures (24)

• Figure 1: Platform architecture.
• Figure 2: LLM integration flow with Ollama LLM-API
• Figure 3: The format of the dataset used to fine-tune the LLMs.
• Figure 4: Fine-tune LLMs with Qlora and deploy with Ollama.
• Figure 5: Prompt for OpenAI-gpt-oss reasoning LLM for final prediction reasoning.