Training-Free Adaptation of New-Generation LLMs using Legacy Clinical Models

TL;DR

The paper addresses the challenge of deploying cutting-edge general-domain LLMs in clinical settings, where domain-specific reliability is critical and retraining for each new model generation is prohibitive. It introduces Cross-Architecture Proxy Tuning (CAPT), a training-free method that re-ranks a new-generation model's top-k token proposals using a contrastive, decode-aware offset derived from an older, domain-specific model, enabling effective integration across disjoint vocabularies. Across six clinical classification and text-generation tasks, CAPT consistently surpasses prior ensembling methods, delivering substantial gains in accuracy and safety metrics and demonstrating clinically meaningful token-level shifts toward domain-specific language. The work offers a practical pathway to leverage modern LLMs in healthcare without costly domain-adaptation per generation, supported by token-level analyses and physician-case studies that highlight improved clinical terminology, specificity, and decision-support content.

Abstract

Adapting language models to the clinical domain through continued pretraining and fine-tuning requires costly retraining for each new model generation. We propose Cross-Architecture Proxy Tuning (CAPT), a model-ensembling approach that enables training-free adaptation of state-of-the-art general-domain models using existing clinical models. CAPT supports models with disjoint vocabularies, leveraging contrastive decoding to selectively inject clinically relevant signals while preserving the general-domain model's reasoning and fluency. On six clinical classification and text-generation tasks, CAPT with a new-generation general-domain model and an older-generation clinical model consistently outperforms both models individually and state-of-the-art ensembling approaches (average +17.6% over UniTE, +41.4% over proxy tuning across tasks). Through token-level analysis and physician case studies, we demonstrate that CAPT amplifies clinically actionable language, reduces context errors, and increases clinical specificity.

Figures (18)

• Figure 1: Mean log-probability offset between $M_{\text{old-clin}}$ and $M_{\text{old}}$ of generated tokens by semantic category. Positive shifts (orange) indicate increased influence of $M_{\text{old-clin}}$, while negative shifts (blue) of $M_{\text{new}}$. All token categories are shown in Appendix \ref{['sec:token_categories']}, Figure \ref{['fig:all_token_categories1']}.
• Figure 2: CAPT-generated post-operative management plan for an MTS-Procedure example involving a forearm arteriovenous graft (connecting an artery to a vein). Orange and blue highlights indicate stronger influence from $M_{\text{old-clin}}$ and $M_{\text{new}}$, respectively. Bolded tokens denote when the top-choice token changed after CAPT adjustment, with the original top-choice crossed out. Green circles mark tokens discussed in main text. The full output is shown in Appendix \ref{['sec:case_study']}, Figure \ref{['fig:case1']}.
• Figure 3: Full version of Figure \ref{['fig:token_categories']} with all token categories shown. Mean log-probability offset between $M_{\text{old-clin}}$ and $M_{\text{old}}$ of generated tokens by semantic category. Positive shifts (orange) indicate increased influence of $M_{\text{old-clin}}$, while negative shifts (blue) of $M_{\text{new}}$.
• Figure 4: Case Study 1 - CAPT Output, LLM Jury Score = 4.22. Majority of comments related to this output can be found in Section \ref{['sec:results']}, Results. We include additional comments here. CAPT increases clinical specificity, resulting in a more clinically actionable and descriptive plan. CAPT heavily preferences "proph" after "Administer," emphasizing the preventative goal of the recommended antibiotic administration. CAPT also replaces "institutional" with "surgical" before "protocol," increasing specificity as typically each institution has a protocol for each surgery. CAPT replaces the generic "clinical exam," which can include physical and imaging exams, with "physical exam", which is more specific to the type of follow-up care the patient needs. CAPT also adds monitoring "thrill," which is an important physical exam function indicating that the graft is functioning. CAPT also uses more clinically relevant terminology. For example, CAPT replaces "optimize" with "resume" when discussing hemodialysis, which is an important distinction as it is not possible to optimize and that language is not used clinically. Additionally, CAPT describes a "native" graft, which is the most common and best way of performing the graft. Despite these improvements, there remain limitations. For example, the plan advises "If the patient is not yet on dialysis, initiate or resume hemodialysis" and to "Assess current dialysis adequacy," which does not provide any patient-specific advice and would not be useful language in a treatment plan. Similarly, the advice for "strict glycemic control (if diabetic)" and to "Continue antihypertensive agents preferentially those safe in CKD" are also not patient-specific and are far more general than would be found in a real patient treatment plan. This is likely an artifact of the task itself, where insufficient patient information (e.g. about prior medications or medical history) is given to provide detailed patient-specific advice, though is worth noting as a weakness.
• Figure 5: Case Study 1 - CAPT Output, LLM Jury Evaluation, LLM Jury Score = 4.22.