MedAide: Leveraging Large Language Models for On-Premise Medical Assistance on Edge Devices

TL;DR

MedAide, an on-premise healthcare chatbot that leverages tiny-LLMs integrated with LangChain, providing efficient edge-based preliminary medical diagnostics and support, enabling an energy-efficient healthcare assistance platform that alleviates privacy concerns due to edge-based deployment, thereby empowering the community.

Abstract

Large language models (LLMs) are revolutionizing various domains with their remarkable natural language processing (NLP) abilities. However, deploying LLMs in resource-constrained edge computing and embedded systems presents significant challenges. Another challenge lies in delivering medical assistance in remote areas with limited healthcare facilities and infrastructure. To address this, we introduce MedAide, an on-premise healthcare chatbot. It leverages tiny-LLMs integrated with LangChain, providing efficient edge-based preliminary medical diagnostics and support. MedAide employs model optimizations for minimal memory footprint and latency on embedded edge devices without server infrastructure. The training process is optimized using low-rank adaptation (LoRA). Additionally, the model is trained on diverse medical datasets, employing reinforcement learning from human feedback (RLHF) to enhance its domain-specific capabilities. The system is implemented on various consumer GPUs and Nvidia Jetson development board. MedAide achieves 77\% accuracy in medical consultations and scores 56 in USMLE benchmark, enabling an energy-efficient healthcare assistance platform that alleviates privacy concerns due to edge-based deployment, thereby empowering the community.

Figures (11)

• Figure 1: Mortality rate for 137 countries due to inadequate healthcare facilities Kruk2018MortalityDT where MedAide could be effective.
• Figure 2: The analysis of embedded devices from NVIDIA, Intel, and Google reveals a significant gap between the capabilities of embedded boards and the requirements of recent LLMs due to escalating memory demands and computational requirements.
• Figure 3: Open-LLM Leader-board benchmark Open-LLM-Leaderboard-Report-2023 competing various state-of-the-art LLMs across diverse benchmarks, encompassing TruthfulQA lin2022truthfulqa, MMLU hendrycks2021measuring, ARC clark2018think, and HellaSwag zellers2019hellaswag for a comprehensive evaluation.
• Figure 4: A comprehensive comparison between selected LLMs (OPT, LLaMa2, and Bloom) and state-of-the-art LLMs (Galactica, Gopher, Chinchilla, and Flan-Palm) to evaluate the performance of these models on different domains, shedding light on their feasibility for medical assistance.
• Figure 5: MedAide System overview with the input system requirements and system processes to generate the outputs.