Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptation

TL;DR

Medical Lay Language Generation (MLLG) faces semantic fidelity and diverse lay-style generation challenges when fine-tuning on heterogeneous datasets. Magical introduces an asymmetric LoRA framework with a shared matrix $A$ for abstractive summarization and multiple per-style matrices $B$ to capture diverse lay styles, augmented by a Semantic Invariance Constraint on $A$ and a Recommendation-guided Switch to select the appropriate $B$. Empirical results across three real-world datasets and multiple backbones show that Magical consistently surpasses prompt-based methods and standard LoRA variants, while reducing trainable parameters by about 31.66%. The approach demonstrates strong semantic alignment and improved lay-language quality, with ablations confirming the importance of semantic constraints and multi-style adaptation. These findings suggest a modular, semantically-aware, style-aware fine-tuning paradigm that can generalize across domains and support more accessible medical narratives.

Abstract

Medical Lay Language Generation (MLLG) plays a vital role in improving the accessibility of complex scientific content for broader audiences. Recent literature to MLLG commonly employ parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA) to fine-tuning large language models (LLMs) using paired expert-lay language datasets. However, LoRA struggles with the challenges posed by multi-source heterogeneous MLLG datasets. Specifically, through a series of exploratory experiments, we reveal that standard LoRA fail to meet the requirement for semantic fidelity and diverse lay-style generation in MLLG task. To address these limitations, we propose Magical, an asymmetric LoRA architecture tailored for MLLG under heterogeneous data scenarios. Magical employs a shared matrix $A$ for abstractive summarization, along with multiple isolated matrices $B$ for diverse lay-style generation. To preserve semantic fidelity during the lay language generation process, Magical introduces a Semantic Invariance Constraint to mitigate semantic subspace shifts on matrix $A$. Furthermore, to better adapt to diverse lay-style generation, Magical incorporates the Recommendation-guided Switch, an externally interface to prompt the LLM to switch between different matrices $B$. Experimental results on three real-world lay language generation datasets demonstrate that Magical consistently outperforms prompt-based methods, vanilla LoRA, and its recent variants, while also reducing trainable parameters by 31.66%. Our code is publicly available at https://github.com/tianlwang/Magical.git.

Figures (7)

• Figure 1: Distribution of Word Count, DCRS goldsack2023biolaysumm and readability evaluation of DeepSeek-V3 on three heterogeneous MLLG Datasets.
• Figure 2: Projections of activations from expert text and lay text on the top-2 singular directions of the semantic subspace, which form the x- and y-axes of the KDE plot.
• Figure 3: Overview of the Magical. (a) Illustrates the target audience of expert-lay language and the heterogeneity of multi-source datasets. (b) Depicts Magical employs probing techniques to identify semantic-relevant layers for subsequent Semantic Contrastive Learning. (c) Shows Magical applies Semantic Contrastive Learning on matrix $A$ to enforce semantic invariance, and utilizes an external Recommendation Agent to switch between different matrices $B$ for Layperson-tailored Adaptation.
• Figure 4: (a) shows Kernel Density Estimation based on the projections of activations obtained with / without the application of Semantic Invariance Constraint on $A$. (b) presents Cross-correlation Matrix for the representations of expert text and lay text under matrix $A$ with / without the use of Semantic Invariance Constraint on $A$.
• Figure 5: Confusion matrix of predictions made by Router selection matrix $B$ for target lay-style.