UniCAD: Efficient and Extendable Architecture for Multi-Task Computer-Aided Diagnosis System

TL;DR

UniCAD addresses the efficiency and scalability challenges of multi-task CAD for diverse 2D/3D medical images by freezing a vision foundation model and injecting lightweight task-specific LoRA adapters through a Unified Embedding Layer. It achieves only $0.17\%$ trainable parameters while enabling seamless 2D/3D processing and random task flow, supported by a rank-standardized multi-task batch processing scheme. Across 12 datasets, UniCAD delivers competitive or superior diagnostic accuracy with favorable memory and throughput characteristics, validated by a novel DEAR metric that remains near 1 as tasks grow. The work also promotes an open ecosystem for sharing lightweight CAD experts, enabling privacy-preserving collaboration and rapid deployment in clinical settings.

Abstract

The growing complexity and scale of visual model pre-training have made developing and deploying multi-task computer-aided diagnosis (CAD) systems increasingly challenging and resource-intensive. Furthermore, the medical imaging community lacks an open-source CAD platform to enable the rapid creation of efficient and extendable diagnostic models. To address these issues, we propose UniCAD, a unified architecture that leverages the robust capabilities of pre-trained vision foundation models to seamlessly handle both 2D and 3D medical images while requiring only minimal task-specific parameters. UniCAD introduces two key innovations: (1) Efficiency: A low-rank adaptation strategy is employed to adapt a pre-trained visual model to the medical image domain, achieving performance on par with fully fine-tuned counterparts while introducing only 0.17% trainable parameters. (2) Plug-and-Play: A modular architecture that combines a frozen foundation model with multiple plug-and-play experts, enabling diverse tasks and seamless functionality expansion. Building on this unified CAD architecture, we establish an open-source platform where researchers can share and access lightweight CAD experts, fostering a more equitable and efficient research ecosystem. Comprehensive experiments across 12 diverse medical datasets demonstrate that UniCAD consistently outperforms existing methods in both accuracy and deployment efficiency. The source code and project page are available at https://mii-laboratory.github.io/UniCAD/.

Figures (7)

• Figure 1: Comparison of different multi-task CAD systems. From left to right: (1) a single model handling multiple tasks, which consumes significant GPU memory and has limited deployment flexibility; (2) multiple independent models for distinct tasks, resulting in high memory consumption and redundancy; and (3) our proposed UniCAD, a unified architecture that combines a single general model with multiple small, task-specific experts, optimizing memory efficiency while maintaining flexibility for diverse tasks.
• Figure 2: Recent trends have shown that the parameter sizes of models in both CV and NLP are rapidly increasing, achieving outstanding performance. This underscores the importance of leveraging these vision foundation models to enhance the capabilities of multi-task CAD systems. "*" means the parameter size is an estimate, rather than the official release.
• Figure 3: The visualization showcases the datasets and task-specific experts used in our experiments. Red boxes represent 2D datasets, while yellow boxes denote 3D datasets. Building on foundation models derived from natural images, we achieve a wide range of diagnostic tasks for the human body at a relatively low cost, while maintaining strong performance across tasks.
• Figure 4: The illustration of overall architecture depicts how images are handled through UniCAD, using 3D fracture, 2D pathology, and chest X-ray as examples in the random task flow. At its core, UniCAD employs a Unified Embedding Layer (UEL) that seamlessly integrates both 2D and 3D medical images within a single framework. This architecture enables adaptable and versatile diagnostic capabilities for diverse clinical applications.
• Figure 5: The attention visualization from different models for 2D and 3D medical images and diagnosis tasks.