Reproduce “Scaling Clinician-Grade Feature Generation from Clinical Notes with Multi-Agent Language Models” on OpenResearch

About this paper

Jiayi Wang, Jacqueline Jil Vallon, Nikhil V. Kotha, Neil Panjwani, Xi Ling, Margaret Redfield, Sushmita Vij, Sandy Srinivas and 3 more

SNOW is a modular large language model workflow that turns clinical notes into features for prediction. It proposes and aligns variables, extracts and validates values, and generates aggregation code, with clinicians reviewing outputs. In Stanford prostate-cancer recurrence prediction, baseline data plus SNOW features reached mean AUC-ROC 0.767, versus 0.762 with manual features; this is methodological evidence, not deployment validation.

Key insights

• Agentic LLM systems can effectively operationalize and mimic the complex, iterative reasoning processes of human clinicians for feature generation from clinical notes.
• Explicitly modeling clinician reasoning and incorporating validation feedback loops within an LLM system enables the autonomous correction of common data inconsistencies (e.g., negation, unit normalization, outliers).
• Separating feature generation into auditable intermediate artifacts (definitions, extraction rules, validation logs, aggregation code) fosters transparency and trust, which is crucial for clinical AI applications.

Summary by alphaXiv.