Reproduce “OraPO: Oracle-educated Reinforcement Learning for Data-efficient and Factual Radiology Report Generation” on OpenResearch

About this paper

Zhuoxiao Chen, Hongyang Yu, Ying Xu, Yadan Luo, Long Duong, Yuan-Fang Li

OraPO, an Oracle-educated Reinforcement Learning framework, combined with a FactScore-based reward, enables data-efficient and factual radiology report generation from chest X-rays. Using only 1,000 training samples, the framework achieved state-of-the-art recall (0.832 on CheXpert Plus) and F1 scores, outperforming methods trained on orders of magnitude more data with smaller models.

Key insights

• Dynamically switching between GRPO and DPO based on exploration success effectively salvages failed explorations and stabilizes reinforcement learning in domain-specific tasks where the initial policy lacks specialized knowledge.
• Supervising factual correctness at an atomic-fact level, rather than relying on superficial text similarity metrics, is crucial for generating clinically reliable long-form reports and significantly boosts performance in medical contexts.
• High-performing and clinically faithful radiology report generation is achievable with significantly reduced data and computational resources by combining efficient reinforcement learning with precise, fact-centric supervision.

Summary by alphaXiv.