Reproduce “Enhancing Accessibility of Medical Texts through Large Language Model-Driven Plain Language Adaptation” on OpenResearch

About this paper

Ting-Wei Chang, Hen-Hsen Huang, Hsin-Hsi Chen

An empirical study of biomedical plain-language adaptation compares zero- and few-shot prompting, semantic example retrieval, QLoRA fine-tuning, and Mixture-of-Agents generation. Semantic demonstrations improve reference alignment, while QLoRA delivers the strongest gains for open-weight models—for example, LLaMA 3 8B reaches BLEU 33.77 and AlignScore 88.82 after fine-tuning—and the fine-tuned MoA system obtains a 0.8238 human evaluation score.

Key insights

• Semantically similar demonstrations generally improve relevance and term-identification performance, although their effect on readability and factuality is limited and inconsistent.
• QLoRA substantially improves content alignment, factuality, completeness, and brevity for open-weight models; the gains are not simply explained by making outputs shorter or easier to read.
• Mixture-of-Agents performance depends strongly on the evaluation method: automatic metrics show limited gains, whereas human assessment favors the version containing fine-tuned agents.

Summary by alphaXiv.